Synthetic training data generation for improved machine learning model generalizability

ABSTRACT

Systems and techniques that facilitate synthetic training data generation for improved machine learning generalizability are provided. In various embodiments, an element augmentation component can generate a set of preliminary annotated training images based on an annotated source image. In various aspects, a preliminary annotated training image can be formed by inserting at least one element of interest or at least one background element into the annotated source image. In various instances, a modality augmentation component can generate a set of intermediate annotated training images based on the set of preliminary annotated training images. In various cases, an intermediate annotated training image can be formed by varying at least one modality-based characteristic of a preliminary annotated training image. In various aspects, a geometry augmentation component can generate a set of deployable annotated training images based on the set of intermediate annotated training images. In various instances, a deployable annotated training image can be formed by varying at least one geometric characteristic of an intermediate annotated training image. In various embodiments, a training component can train a machine learning model on the set of deployable annotated training images.

TECHNICAL FIELD

The subject disclosure relates generally to training of machine learningmodels, and more specifically to synthetic training data generation forimproved machine learning model generalizability.

BACKGROUND

The efficacy and/or generalizability of a machine learning model dependsupon the veracity, volume, variety, and/or velocity of the data on whichthe machine learning model is trained. In other words, theimplementation of high quality, more voluminous, more varied/diverse,and/or more readily available training data can result in the creationof machine learning models that are invariant to various challengesfaced in real-world operational scenarios. Conversely, theimplementation of low quality, less voluminous, less varied/diverse,and/or less readily available training data can result in the creationof machine learning models that are easily impeded by various challengesfaced in real-world operational scenarios. Thus, systems and/ortechniques that can increase the veracity, volume, variety, and/orvelocity of available training data can be desirable.

SUMMARY

The following presents a summary to provide a basic understanding of oneor more embodiments of the invention. This summary is not intended toidentify key or critical elements, or delineate any scope of theparticular embodiments or any scope of the claims. Its sole purpose isto present concepts in a simplified form as a prelude to the moredetailed description that is presented later. In one or more embodimentsdescribed herein, devices, systems, computer-implemented methods,apparatus and/or computer program products that facilitate synthetictraining data generation for improved machine learning modelgeneralizability are provided.

According to one or more embodiments, a system is provided. The systemcan comprise a memory that can store computer-executable components. Thesystem can further comprise a processor that can be operably coupled tothe memory and that can execute the computer-executable componentsstored in the memory. In various embodiments, the computer-executablecomponents can comprise an element augmentation component that cangenerate a set of preliminary annotated training images based on anannotated source image. In various aspects, a preliminary annotatedtraining image can be formed by inserting at least one element ofinterest or at least one background element into the annotated sourceimage. In various instances, the computer-executable components cancomprise a modality augmentation component that can generate a set ofintermediate annotated training images based on the set of preliminaryannotated training images. In various cases, an intermediate annotatedtraining image can be formed by varying at least one modality-basedcharacteristic of a preliminary annotated training image. In variousaspects, the computer-executable components can comprise a geometryaugmentation component that can generate a set of deployable annotatedtraining images based on the set of intermediate annotated trainingimages. In various instances, a deployable annotated training image canbe formed by varying at least one geometric characteristic of anintermediate annotated training image. In various embodiments, thecomputer-executable components can comprise a training component thatcan train a machine learning model on the set of deployable annotatedtraining images.

According to one or more embodiments, the above-described system can beimplemented as a computer-implemented method and/or a computer programproduct.

According to one or more embodiments, a computer program product can beprovided. In various cases, the computer program product can comprise acomputer readable memory having program instructions embodied therewith.In various cases, the program instructions can be executable by theprocessor to cause the processor to perform various operations. In someinstances, such operations can comprise parametrizing a simulation spaceof data segments by defining a set of augmentation subspaces, whereineach augmentation subspace comprises a corresponding set of augmentableparameters. In various instances, each augmentable parameter can have acorresponding parametric range of possible values or states. In variousaspects, the operations can further comprise receiving a source datasegment. In various embodiments, the operations can further comprise,for each augmentable parameter, sampling a parametric range of possiblevalues or states corresponding to the augmentable parameter. In somecases, this can yield a collection of sampled ranges of values or statesthat represents the simulation space. In various aspects, the operationscan further comprise generating a set of training data segments byapplying the collection of sampled ranges of values or states to copiesof the source data segment.

DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 illustrates a block diagram of an example, non-limiting systemthat facilitates synthetic training data generation for improved machinelearning model generalizability in accordance with one or moreembodiments described herein.

FIG. 2 illustrates a block diagram of an example, non-limiting systemincluding an element catalog that facilitates synthetic training datageneration for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

FIGS. 3-4 illustrate block diagrams of example, non-limiting preliminarytraining images formed from an annotated source image in accordance withone or more embodiments described herein.

FIG. 5 illustrates a block diagram of an example, non-limiting systemincluding modality-based characteristics that facilitates synthetictraining data generation for improved machine learning modelgeneralizability in accordance with one or more embodiments describedherein.

FIG. 6 illustrates a block diagram of example, non-limiting intermediatetraining images formed from preliminary training images in accordancewith one or more embodiments described herein.

FIG. 7 illustrates a block diagram of an example, non-limiting systemincluding geometric transformations that facilitates synthetic trainingdata generation for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

FIG. 8 illustrates a block diagram of example, non-limiting deployabletraining images formed from intermediate training images in accordancewith one or more embodiments described herein.

FIG. 9 illustrates a block diagram of example, non-limiting variationsof modality-based characteristics and geometric characteristics inaccordance with one or more embodiments described herein.

FIG. 10 illustrates example, non-limiting experimental results inaccordance with one or more embodiments described herein.

FIGS. 11-20 illustrate block diagrams of example, non-limiting imageaugmentations in accordance with one or more embodiments describedherein.

FIG. 21 illustrates a flow diagram of an example, non-limitingcomputer-implemented method that facilitates synthetic training datageneration for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

FIG. 22 illustrates a flow diagram of an example, non-limitingcomputer-implemented method that facilitates synthetic training datageneration for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

FIG. 23 illustrates a block diagram of an example, non-limitingaugmentation space hierarchy that facilitates synthetic training datageneration for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

FIG. 24 illustrates a block diagram of an example, non-limitingoperating environment in which one or more embodiments described hereincan be facilitated.

FIG. 25 illustrates an example networking environment operable toexecute various implementations described herein.

DETAILED DESCRIPTION

The following detailed description is merely illustrative and is notintended to limit embodiments and/or application or uses of embodiments.Furthermore, there is no intention to be bound by any expressed orimplied information presented in the preceding Background or Summarysections, or in the Detailed Description section.

One or more embodiments are now described with reference to thedrawings, wherein like referenced numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea more thorough understanding of the one or more embodiments. It isevident, however, in various cases, that the one or more embodiments canbe practiced without these specific details.

A machine learning model can be any suitable artificial intelligencemodel and/or algorithm that can map a set (e.g., one or more) of inputvariables to a set (e.g., one or more) of output variables. In variousaspects, each of the output variables can be referred to as a class, aclassification, a label, a category, a segmentation, a detection, and/orso on. In other words, a machine learning model can receive input dataand can determine to which class the input data belongs (e.g., canclassify the input data). In other cases, a machine learning model canproduce as output any suitable segmentations, determinations, decisions,predictions, inferences, regressions, and/or so on. In various aspects,a machine learning model can be designed and/or configured to receiveany suitable type of input data of any suitable dimensionality (e.g.,scalars, vectors, matrices, and/or tensors) and to generate any suitabletype of output data of any suitable dimensionality (e.g., scalars,vectors, matrices, and/or tensors). As some non-limiting examples, amachine learning model can be configured to perform image recognition,classification, and/or segmentation (e.g., recognizing characterstrings, numeric objects, and/or alphanumeric objects depicted inimages; recognizing flora and/or fauna depicted in images; recognizinganatomical structures depicted in images; recognizing inanimate objectsdepicted in images), can be configured to perform sound recognition,classification, and/or segmentation (e.g., recognizing spoken letters,words, and/or speech present in audio data; recognizing voices presentin audio data; recognizing sounds of fauna present in audio data;recognizing sounds of inanimate objects present in audio data), and/orany other suitable type of data recognition, classification,segmentation, prediction, determination, and/or detection (e.g.,distinguishing spam emails from non-spam emails; distinguishingcustomers that are likely to transact from customers that are unlikelyto transact; distinguishing transactions that are likely fraudulent fromtransactions that are unlikely fraudulent; and/or so on). In variousaspects, any suitable output dimensionality can be implemented (e.g.,binary classifications, tertiary classifications, quaternaryclassifications, and/or any suitable higher-order classifications).

In various aspects, a machine learning model can be trained (e.g., viasupervised training, unsupervised training, and/or reinforcementlearning) to classify, label, and/or make any other determinations,predictions, and/or inferences about received input data. Whensupervised training is implemented, each piece of training data can havea corresponding annotation. In various aspects, the correspondingannotation can indicate the true classification to which the piece oftraining data is known to belong (e.g., can represent a ground truth).During supervised training, the machine learning model can be fed apiece of training data, and the machine learning model can accordinglygenerate a resulting classification. In various cases, a differencebetween the resulting classification and the known annotation can beused (e.g., in back propagation) to update parameters of the machinelearning model. Updating the parameters of the machine learning model inthis way can help to cause the machine learning model to more accuratelyanalyze future input data that is similar to the training data.

In various instances, the efficacy of a machine learning model candepend upon the quality of the training which the machine learning modelundergoes. In other words, a machine learning model can perform better(e.g., can more accurately analyze input data) when the machine learningmodel is trained on better and/or higher quality training data. Invarious aspects, the quality of training data can be described in termsof veracity, volume, variety, and/or velocity. In various instances, theveracity of the training data can relate to the accuracy of the knownannotations that correspond to the training data (e.g., parameters of amachine learning model can be accurately updated/adjusted only whenaccurate annotations of training data are involved; so, if the knownannotations of the training data are not accurate, the training can beineffective, and the machine learning model can fail to accuratelyanalyze input data when the machine learning model is deployed inreal-life). In various cases, the volume of the training data can relateto the amount of the training data (e.g., parameters of a machinelearning model can be more fully/appropriately updated/adjusted whenmore training data is available; so, if little training data isavailable to feed to the machine learning model, the training can beineffective, and the machine learning model can fail to accuratelyanalyze input data when the machine learning model is deployed inreal-life). In various aspects, the variety of the training data canrelate to the feature diversity that is present within the training data(e.g., a machine learning model can be trained to detect and/or ignoreonly those features that are present within the training data; so, ifthere is not much real-world variety in the features depicted in thetraining data, the training can be ineffective, and the machine learningmodel can fail to accurately analyze input data when the machinelearning model is deployed in real-life). In some instances, thevelocity of the training data can relate to how quickly the trainingdata and associated annotations can be collected from training datasources (e.g., a machine learning model can be trained only whenannotated training data is available; so, if it takes days, weeks, ormonths to generate annotated training data, waiting days, weeks, ormonths can be required before training and/or deploying the machinelearning model).

In short, inadequate training data can lead to inadequate machinelearning models (e.g., there can be a 5% to 40% drop in performanceaccuracy when a model is operated on a dataset not represented by thetraining dataset). Thus, improving the veracity, volume, variety, and/orvelocity of training data can help to improve the generalizability of amachine learning model. For example, a machine learning model that istrained on high veracity, high volume, high variety, and/or highvelocity training data can accurately analyze input data regardless ofreal-world variability in the input data. Conversely, a machine learningmodel that is trained on low veracity, low volume, low variety, and/orlow velocity training data can be easily thrown off by real-worldvariability in input data and thus can fail to accurately analyze theinput data. In various aspects, systems and/or techniques that canimprove veracity, volume, variety, and/or velocity of training data canthus be desirable.

Various embodiments of the subject innovation can address one or more ofthese issues/problems. One or more embodiments described herein includesystems, computer-implemented methods, apparatus, and/or computerprogram products that can facilitate synthetic training data generationfor improved machine learning model generalizability. In variousinstances, embodiments of the subject innovation can be considered ascomputerized tools for quickly generating veracious, voluminous, and/orvaried training data for any suitable machine learning model. In variousaspects, embodiments of the subject innovation can then train a machinelearning model on the quickly generated, veracious, voluminous, and/orvaried training data, thereby improving the efficacy and/orgeneralizability of the machine learning model.

For ease of explanation, the herein teachings regarding the quickgeneration of veracious, voluminous, and/or varied training data arediscussed in relation to machine learning models that are configured toclassify/label two-dimensional medical images in clinical contexts.However, it should be understood that this is exemplary andnon-limiting. In various aspects, the herein teachings can be used toquickly generate veracious, voluminous, and/or varied training data forany suitable machine learning model that is configured to generate anysuitable type of result (e.g., classification, segmentation,determination, inference, prediction, and/or so on) in any suitableoperational context (e.g., machine learning models that are configuredto receive two-dimensional and/or three-dimensional image data as input,machine learning models that are configured to receive one-dimensionaland/or multi-dimensional sound data as input, and/or machine learningmodels that are configured to receive any other suitable data having anysuitable dimensionality as input).

In various instances, embodiments of the subject innovation canelectronically receive an annotated source image. In various aspects,the annotated source image can be a medical image of a patient (e.g.,X-ray image of the patient, computed tomography (CT) image of thepatient, magnetic resonance imaging (MRI) image of the patient, positronemission tomography (PET) image of the patient, visible-light-spectrumphotograph of the patient, and/or so on). In various aspects, theannotated source image can be generated and/or captured by any suitableimaging device and/or apparatus, and the annotated source image can bereceived directly from the imaging device and/or apparatus. In variousother aspects, the annotated source image can be stored in any suitabledatabase and/or data structure, and the annotated source image can beretrieved from the database and/or data structure. In various cases, theannotated source image, as its name implies, can be associated with anannotation. In various aspects, the annotation can be any suitableindication of a class, classification, category, and/or label that isknown to apply to the annotated source image (e.g., the annotation canindicate that the annotated source image depicts a patient with a brainlesion, the annotation can indicate that the annotated source imagedepicts a patient with tooth decay, the annotation can indicate that theannotated source image depicts a patient with a clogged blood vessel,the annotation can indicate that the annotated source image depicts apatient with a particular skin condition, the annotation can indicatethat the annotated source image depicts a patient with lung cancer,and/or so on). In various aspects, the annotation can be at any suitablelevel of granularity and/or specificity (e.g., the annotation canindicate merely the condition afflicting the patient, and/or theannotation can more specifically indicate any other information thatcharacterizes the condition afflicting the patient, such aslocalization/laterality of the condition, severity of the condition, ageof the condition, prognosis associated with the condition, and/or soon). In various instances, the annotation can be generated and/orcreated by any suitable technique, such as manually by a clinicianand/or medical professional.

As described herein, various embodiments of the subject innovation canelectronically receive the annotated source image and can electronicallygenerate a plurality of veracious, voluminous, and/or varied trainingimages based on the annotated source image. In various aspects, this canbe accomplished by copying the annotated source image and by performingthree different types of augmentations on the copies of the annotatedsource image.

Specifically, in various instances, embodiments of the subjectinnovation can generate, via an element augmentation component, a set ofpreliminary annotated training images (e.g., also referred to aspreliminary training images) based on the annotated source image. Invarious aspects, a preliminary annotated training image can be formed byinserting into the annotated source image at least one element/featureof interest and/or at least one background element/feature. In variousaspects, an element/feature of interest (at least with respect toimages) can be any suitable visual object and/or visual characteristicwhich can be added to the annotated source image (e.g., which can beadded to a copy of the annotated source image) and which is anelement/feature that the machine learning model to be trained issupposed to learn, predict, detect, and/or classify. For example, if themachine learning model to be trained is supposed to learn, predict,detect, and/or classify different types of brain lesions, anelement/feature of interest can be a stand-alone image of a particularbrain lesion that is insertable into the annotated source image. Asanother example, if the machine learning model to be trained is supposedto learn, predict, detect, and/or classify skin growths, anelement/feature of interest can be a stand-alone image of a particularskin growth that is insertable into the annotated source image. In somecases, an element/feature of interest can be referred to as a positiveelement/feature. In various aspects, a background element/feature (atleast with respect to images) can be any suitable visual object and/orvisual characteristic which can be added to the annotated source image(e.g., which can be added to a copy of the annotated source image) andwhich is an element/feature that the machine learning model to betrained is not supposed to learn, predict, detect, and/or classify.Instead, in various cases, a background element/feature can negativelyaffect classifications generated by the machine learning model (e.g.,can distract and/or throw off the machine learning model). For example,if the machine learning model to be trained is supposed to learn,predict, detect, and/or classify a particular type of morbidity, abackground element/feature can be a stand-alone image of an unrelatedco-morbidity that is insertable into the annotated source image. Asanother example, in some cases, a background element/feature can be astand-alone image of a particular piece of medical equipment that isinsertable into the annotated source image. In various cases, when anelement/feature is inserted into a copy of the annotated source image,the copy can now be referred to as a preliminary training image. Invarious aspects, any suitable number of preliminary training images canbe generated based on the annotated source image.

In various instances, different elements/features can be differentlylocalized and/or positioned within the annotated source image (e.g.,within a copy of the annotated source image) in different ways, therebyyielding different preliminary training images. In some cases, insertedelements/features can be randomly localized and/or positioned in theannotated source image within any suitable range ofbiologically-possible locations/positions. For instance, suppose thatthe annotated source image is an X-ray of a patient's chest and abdomen.Accordingly, the annotated source image can depict the ribcage of thepatient, the chest cavity of the patient, the intestinal/abdominalcavity of the patient, and/or so on. Further, suppose that the machinelearning model to be trained is supposed to learn, predict, detect,and/or classify lung cancers (and/or to otherwise perform lungsegmentation). In various aspects, an element/feature can be added toand/or inserted into the annotated source image in anybiologically-possible location/position. For example, suppose that theelement/feature is a cancerous lung growth (e.g., an element/feature ofinterest). In various aspects, a first copy of the X-ray can be made,and the cancerous lung growth can be inserted into the first copy in anysuitable location within the depicted chest cavity and can be notinserted into the depicted abdominal cavity (e.g., a cancerous lunggrowth can possibly form in the lungs and thus in the chest cavity of apatient; however, a cancerous lung growth cannot possibly form in theabdominal cavity of the patient). In various cases, the first copy cannow be considered as a first preliminary training image. As anotherexample, suppose that the element/feature is stomach gas (e.g., abackground element/feature). In various aspects, a second copy of theX-ray can be made, and the stomach gas can be inserted into the secondcopy in any suitable location within the depicted abdominal cavity andcan be not inserted into the depicted chest cavity (e.g., stomach gascan possibly form in the abdominal cavity of a patient; however, stomachgas cannot possibly form in the chest cavity of the patient). In thisway, an insertable element/feature can be localized in any suitable,biologically-possible location/position in the annotated source image.In various cases, the second copy can now be considered as a secondpreliminary training image. Thus, different preliminary training imagescan be formed by inserting different elements/features into theannotated source image (e.g., into copies of the annotated sourceimage).

In various aspects, a same element/feature can be differently localizedwithin the annotated source image (e.g., within copies of the annotatedsource image), thereby yielding different preliminary training images.For instance, consider again the example above where the firstpreliminary training image includes an inserted cancerous lung growth.Suppose that the cancerous lung growth is inserted in a top portion of aright lung in the depicted chest cavity of the patient. In some cases, athird copy of the X-ray can be made, and the cancerous lung growth canbe inserted into a bottom portion of a left lung in the depicted chestcavity. In various aspects, the third copy can now be considered as athird preliminary training image. Thus, both the first preliminarytraining image and the third preliminary training image can be formed byinserting the image of the cancerous lung growth into the annotatedsource image, but they can be different preliminary training imagesbecause the cancerous lung growth can be localized differently in each.As another instance, consider again the example above where the secondpreliminary training image includes inserted stomach gas. Suppose thatthe stomach gas is inserted in a top-left portion of the depictedabdominal cavity of the patient. In some cases, a fourth copy of theX-ray can be made, and the stomach gas can be inserted into abottom-right portion in the depicted abdominal cavity. In variousaspects, the fourth copy can now be considered as a fourth preliminarytraining image. Thus, both the second preliminary training image and thefourth preliminary training image can be formed by inserting the imageof the stomach gas into the annotated source image, but they can bedifferent preliminary training images because the stomach gas can belocalized differently in each. In this way, a same element/feature canbe inserted into different copies of the annotated source image indifferent places/locations/positions, thereby yielding differentpreliminary training images.

It should be appreciated that any suitable characteristic of aninsertable element/feature can be varied when inserting theelement/feature into the annotated source image (e.g., into copies ofthe annotated source image). For instance, a same element/feature can beinserted into two different copies of the annotated source image, suchthat the same element/feature has different spatial dimensions (e.g.,length, width, height, thickness), different spatial orientations (e.g.,oriented upside-down, oriented sideways, oriented backwards), and/ordifferent intensities in the different copies of the annotated sourceimage.

Note that, when background elements/features are inserted, thepreliminary training images generated based on the annotated sourceimage can, in some cases, share the annotation of the annotated sourceimage (e.g., the classification and/or label of a preliminary trainingimage can be the same as the classification and/or label of theannotated source image when a background element/feature is inserted).For instance, suppose that the annotated source image depicts a chestX-ray of a patient, and suppose that the annotation indicates that thepatient suffers from pneumonia. In such case, inserting backgroundelements/features (e.g., stomach gas, medical cables/tubes/wires, apacemaker, and/or so on) does not change the fact that the patientsuffers from pneumonia.

Note that, when elements/features of interest are inserted, thepreliminary training images generated based on the annotated sourceimage can, in some cases, have annotations that are based on theinserted elements/features of interest. For instance, suppose that theannotated source image depicts a head CT scan of a patient, and supposethat the annotation indicates that the patient suffers from a left-sideoccluded blood vessel. In such case, inserting elements/features ofinterest can require a commensurate change/update in the annotation. Forinstance, if another occluded blood vessel is inserted in a right sideof a depicted cranial cavity, the annotation can be updated to indicatethat there are both left-side and right-side occluded blood vessels inthe updated image.

Thus, in various embodiments, an annotation of any preliminary trainingimage can be known/created based on the annotation of the annotatedsource image and/or based on the elements/features inserted into theannotated source image.

In various aspects, a catalog of premade, pre-drawn, pre-illustrated,and/or pre-generated elements/features can be maintained, and anysuitable number of premade, pre-drawn, pre-illustrated, and/orpre-generated elements/features from the catalog can be inserted intothe annotated source image (e.g., into copies of the annotated sourceimage) in any suitable locations and/or any suitable orientations togenerate the set of preliminary training images. In various aspects, thecatalog can be any suitable database and/or data structure (e.g.,relational database, graph database, hybrid database). In variousaspects, the elements/features stored in the catalog can be created viaany suitable technique (e.g., the elements/features stored in thecatalog can be electronic copies of hand-drawn elements/features, can beelectronic images of two-dimensional computer-aided-design models, canbe two-dimensional computer-aided-design models themselves, can betwo-dimensional projections of three-dimensional computer-aided-designmodels, can be three-dimensional computer-aided-design modelsthemselves, and/or so on).

In the medical context, such permutatory insertion of elements/featurescan help to more fully simulate and/or approximate real-world biologicalvariability (e.g., a single X-ray scan can fail to adequately representthe full space of biological variability experienced by real-worldpatients; so, to help simulate and/or span the full space of biologicalvariability experienced by real-world patients, various biologicalstructures and/or medical equipment structures can be added to and/orsuperimposed on the single X-ray scan and/or copies of the single X-rayscan).

In various instances, embodiments of the subject innovation cangenerate, via a modality augmentation component, a set of intermediateannotated training images (e.g., also referred to as intermediatetraining images) based on the set of preliminary training images. Invarious aspects, an intermediate training image can be formed by varyingat least one modality-based characteristic of a preliminary trainingimage. In various aspects, a modality-based characteristic (at leastwith respect to images) can be any suitable image property that dependsupon the device modality (e.g., the image-capture device) that generatedand/or captured the annotated source image. For example, differentimage-capture device modalities can exhibit different gamma/radiationlevels, different brightness/contrast levels, different motion/blurlevels, different noise levels, different resolutions, different fieldsof view, different magnification levels, different visual textures,different imaging artifacts (e.g., glares; scratches, dust, and/or anyother obscuring material on a camera lens), and/or so on. It should beunderstood that, in various aspects, some modality-based characteristicscan vary continuously, while other modality-based characteristics canvary discretely. In various instances, an intermediate training imagecan be formed from a preliminary training image by changing thegamma/radiation level, the brightness/contrast level, the motion/blurlevel, the noise level, the resolution, the field of view, themagnification level, the visual texture, and/or the imaging artifacts ofthe preliminary training image. In various cases, any suitable number ofintermediate training images can be formed from each preliminarytraining image by varying one or more modality-based characteristics ofthe preliminary training image. For instance, consider a preliminarytraining image (e.g., one among many generated from the annotated sourceimage) exhibiting an existing gamma/radiation level. In various aspects,a first copy of the preliminary training image can be made, and theexisting gamma/radiation level of the first copy can be changed to afirst gamma/radiation level. In various cases, the first copy of thepreliminary training image can now be considered a first intermediatetraining image. In various aspects, a second copy of the preliminarytraining image can be made, and the existing gamma/radiation level ofthe second copy can be changed to a second gamma/radiation level. Invarious cases, the second copy of the preliminary training image can nowbe considered a second intermediate training image. As another example,suppose that the preliminary training image exhibits an existingbrightness/contrast level. In various aspects, a third copy of thepreliminary training image can be made, and the existingbrightness/contrast level of the third copy can be changed to a firstbrightness/contrast level. In various cases, the third copy of thepreliminary training image can now be considered a third intermediatetraining image. In various aspects, a fourth copy of the preliminarytraining image can be made, and the existing brightness/contrast levelof the fourth copy can be changed to a second brightness/contrast level.In various cases, the fourth copy of the preliminary training image cannow be considered a fourth intermediate training image. As yet anotherexample, suppose that the preliminary training image exhibits anexisting glare. In various aspects, a fifth copy of the preliminarytraining image can be made, and the existing glare of the fifth copy canbe removed, supplemented, and/or changed to a first glare. In variouscases, the fifth copy of the preliminary training image can now beconsidered a fifth intermediate training image. In various aspects, asixth copy of the preliminary training image can be made, and theexisting glare of the sixth copy can be removed, supplemented, and/orchanged to a second glare. In various cases, the sixth copy of thepreliminary training image can now be considered a sixth intermediatetraining image. In this way, any suitable number of intermediatetraining images can be generated by varying in permutatory fashion atleast one modality-based characteristic of each of the preliminarytraining images. In various cases, any suitable policy and/or scheme forvarying modality-based characteristics of preliminary training imagescan be implemented.

In the medical context, such permutatory variation of modality-basedcharacteristics can help to more fully simulate and/or approximatereal-world device modality variability (e.g., a single X-ray scan can begenerated by a single type/model of X-ray machine, and can thus fail toadequately represent the full space of X-ray machine variability presentin real-world medical/clinical environments; so, to help simulate and/orspan the full space of X-ray machine variability present in real-worldmedical/clinical environments, various modality-based characteristics ofthe single X-ray scan and/or copies of the single X-ray scan can beadjusted/changed).

In various aspects, embodiments of the subject innovation can generate,via a geometry augmentation component, a set of deployable annotatedtraining images (e.g., also referred to as deployable training images)based on the set of intermediate training images. In various aspects, adeployable training image can be formed by applying at least onegeometric transformation to an intermediate training image. In variousaspects, a geometric transformation (at least with respect to images)can be any suitable mathematical transformation and/or operation thatcan spatially alter and/or transform an image pixel grid. For example, ageometric transformation can include reflecting an image about anysuitable axis, rotating an image about any suitable axis, cropping anysuitable portion of an image, panning an image, tilting an image,zooming in and/or out on an image, applying an affine and/or elastictransformation to an image, distorting an image away from rectilinearprojection, and/or so on. In various instances, a deployable trainingimage can be formed from an intermediate training image by flipping,rotating, cropping, panning, tilting, zooming, and/or distorting theintermediate training image. In various cases, any suitable number ofdeployable training images can be formed from each intermediate trainingimage by applying one or more geometric transformations to theintermediate training image. For instance, consider an intermediatetraining image (e.g., one among many generated from the intermediatetraining images) exhibiting an existing orientation. In various aspects,a first copy of the intermediate training image can be made, and theexisting orientation of the first copy can be reflected, rotated,panned, tilted, and/or zoomed to a first orientation. In various cases,the first copy of the intermediate training image can now be considereda first deployable training image. In various aspects, a second copy ofthe intermediate training image can be made, and the existingorientation of the second copy can be reflected, rotated, panned,tilted, and/or zoomed to a second orientation. In various cases, thesecond copy of the intermediate training image can now be considered asecond deployable training image. As another example, suppose that theintermediate training image exhibits an existing appearance. In variousaspects, a third copy of the intermediate training image can be made,and the existing appearance of the third copy can be distorted via afirst affine and/or elastic transformation. In various cases, the thirdcopy of the intermediate training image can now be considered a thirddeployable training image. In various aspects, a fourth copy of theintermediate training image can be made, and the existing appearance ofthe fourth copy can be distorted via a second affine and/or elastictransformation. In various cases, the fourth copy of the intermediatetraining image can now be considered a fourth deployable training image.In this way, any suitable number of intermediate training images can begenerated by varying at least one modality-based characteristic of eachof the preliminary training images. In various cases, any suitablepolicy and/or scheme for applying geometric transformations tointermediate training images can be implemented.

In the medical context, such permutatory application of geometrictransformations can help to more fully simulate and/or approximatereal-world image variability (e.g., a single X-ray scan can have certaingeometric characteristics, and can thus fail to adequately represent thefull space of X-ray characteristics present in real-worldmedical/clinical environments; so, to help simulate and/or span the fullspace of X-ray characteristics present in real-world medical/clinicalenvironments, various geometric transformations can be applied to thesingle X-ray scan and/or copies of the single X-ray scan).

In various instances, embodiments of the subject innovation can train,via a training component, a machine learning model on the set ofdeployable training images. Note that, as described herein, a singleannotated source image can be used to automatically and quickly generatea plurality of deployable training images. Specifically, the pluralityof deployable training images can be formed by making different copiesof the annotated source image, by inserting into the different copiesdifferent elements/features in different locations/orientations, byaltering different modality-based characteristics of the differentcopies, by differently altering same modality-based characteristics ofthe different copies, and/or by applying different combinations ofgeometric transformations to the different copies. In other words, asingle annotated source image can be used to create a plurality ofsynthetically-generated training images that help to account forreal-world variability through element/feature insertion, throughmodality-based modulation, and/or through geometric transformations(e.g., there can exist many permutations of different insertableelements/features, different insertion locations and/or orientationsand/or dimensions, different modality-based characteristics, and/ordifferent geometric transformations). Thus, training a machine learningmodel on the plurality of deployable training images can improveperformance and/or efficacy of the machine learning model as compared totraining on the single annotated source image alone.

To help clarify some of the above discussion, consider the followingnon-limiting example. Suppose that it is desired to train a machinelearning model on an initial training dataset. Further, suppose that theinitial training dataset includes an annotated chest X-ray image (e.g.,source image) that is received from an X-ray machine, and suppose thatthe machine learning model is supposed to learn, predict, detect, and/orclassify lung cancer in chest X-ray images. In various aspects, a set ofpreliminary training X-ray images can be formed based on insertingvarious elements/features into the annotated chest X-ray image. Forinstance, in some cases, a first preliminary training X-ray image can beformed by inserting an image of a pacemaker in a heart-location of theannotated chest X-ray image, a second preliminary training X-ray imagecan be formed by placing an image of medical tubing in an upper-leftportion of the annotated chest X-ray image, a third preliminary trainingX-ray image can be formed by inserting the image of the medical tubingin an upper-right portion of the annotated chest X-ray image (e.g., sameelement orientation, different location), a fourth preliminary trainingX-ray image can be formed by inserting a differently oriented/sizedimage of the medical tubing in an upper-left portion of the annotatedchest X-ray image (e.g., different element orientation/dimensions, samelocation), a fifth preliminary training X-ray image can be formed byinserting an image of stomach gas into a lower portion of the annotatedchest X-ray image, and a sixth preliminary training X-ray image can beformed by inserting no elements/features into the annotated chest X-ray.That is, in various cases, element/feature insertion can be implementedto generate six preliminary training X-ray images based on the singleannotated X-ray image.

In various aspects, a set of intermediate training X-ray images can beformed based on adjusting various modality-based characteristics of eachof the six preliminary training X-ray images. For instance, in somecases, suppose that there are three possible gamma/radiation levelswhich can be exhibited in an X-ray image (e.g., high gamma/radiation,medium gamma/radiation, low gamma/radiation), suppose that there arethree possible blur levels which can be exhibited in an X-ray image(e.g., high blur, medium blur, low blur), and suppose that there are twopossible artifacts which can be exhibited in an X-ray image (e.g., lensglare vs. no lens glare). In such case, eighteen intermediate trainingX-ray images can be formed from each of the preliminary training X-rayimages (e.g., three gamma/radiation levels multiplied by three blurlevels multiplied by two artifact levels), for a total of one hundredeight intermediate training X-ray images (e.g., eighteen intermediatetraining X-ray images per preliminary training X-ray image multiplied bysix preliminary training X-ray images).

In various aspects, a set of deployable training X-ray images can beformed based on applying various geometric transformations to each ofthe intermediate training X-ray images. For instance, in some cases,suppose that available geometric transformations include three potentialreflections (e.g., reflecting about a horizontal axis, reflecting abouta vertical axis, and/or not reflecting at all), four potential rotations(e.g., rotating clockwise by 15 degrees, rotating clockwise by 45degrees, rotating clockwise by 75 degrees, and/or not rotating at all),two possible crops (e.g., applying a central crop vs. not applying acentral crop), and two possible distortions (e.g., applying a barreldistortion vs. not applying a barrel distortion). In such case,forty-eight different deployable training X-ray images can be formedfrom each intermediate training X-ray image (e.g., three possiblereflections multiplied by four possible rotations multiplied by twopossible crops multiplied by two possible distortions), for a total of5,184 deployable training X-ray images (e.g., forty-eight deployabletraining X-ray images per intermediate training X-ray image multipliedby one hundred eight intermediate training X-ray images). That is, byapplying the teachings disclosed herein, a single annotated X-ray imagein the initial training dataset can be leveraged to syntheticallygenerate very many (e.g., 5,184) deployable training X-ray images whichsimulate real-world variety and on which the machine learning model canbe trained. If the initial training dataset includes one hundredannotated X-ray images instead of just one, various embodiments of thesubject innovation can thus generate 518,400 deployable training X-rayimages (e.g., 5,184 deployable training X-ray images per annotated X-rayimage in the initial training dataset multiplied by 100 annotated X-rayimages in the initial training dataset). In various aspects, trainingthe machine learning model on the set of deployable training X-rayimages can yield significantly improved efficacy and/or performance ascompared to training the machine learning model only on the initialtraining dataset. Indeed, by inserting various elements/features, byvarying different modality-based characteristics, and/or by applyingdifferent geometric transformations, embodiments of the subjectinnovation can electronically create a set of training data which cancause a machine learning model to become invariant to and/or robustagainst such various elements/features, different modality-basedcharacteristics, and/or different geometric transformations.

It should be appreciated that the numbers and/or details in the aboveexample are exemplary, non-limiting, and for purposes of illustration.

Various embodiments of the subject innovation can be employed to usehardware and/or software to solve problems that are highly technical innature (e.g., to facilitate synthetic training data generation forimproved machine learning model generalizability), that are not abstractand that cannot be performed as a set of mental acts by a human.Further, some of the processes performed can be performed by aspecialized computer (e.g., trained machine learning model) for carryingout defined tasks related to synthetic training data generation forimproved machine learning model generalizability (e.g., generating a setof preliminary annotated training images based on an annotated sourceimage, wherein a preliminary annotated training image is formed byinserting at least one element of interest or at least one backgroundelement into the annotated source image; generating a set ofintermediate annotated training images based on the set of preliminaryannotated training images, wherein an intermediate annotated trainingimage is formed by varying at least one modality-based characteristic ofa preliminary annotated training image; generating a set of deployableannotated training images based on the set of intermediate annotatedtraining images, wherein a deployable annotated training image is formedby varying at least one geometric characteristic of an intermediateannotated training image; and training a machine learning model on theset of deployable annotated training images). Such defined tasks are notconventionally performed manually by humans. Moreover, neither the humanmind nor a human with pen and paper can electronically insertelements/features into an image, can electronically vary modality-basedcharacteristics of an image, or can electronically adjust geometriccharacteristics of an image. Instead, various embodiments of the subjectinnovation are inherently and inextricably tied to computer technologyand cannot be implemented outside of a computing environment (e.g.,embodiments of the subject innovation constitute a computerized devicethat synthetically generates many varied training images based on agiven annotated source image; such a computerized device can exist onlyin a computing environment).

In various instances, embodiments of the invention can integrate into apractical application the disclosed teachings regarding synthetictraining data generation for improved machine learning modelgeneralizability. Indeed, in various embodiments, the disclosedteachings can provide a computerized system that receives as input oneor more annotated source images (e.g., real-world medical/clinicalimages of patients that have associated annotations created byreal-world medical/clinical professionals) and that produces as output aplurality of training images based on the one or more annotated sourceimages, where the plurality of training images are formed by copying theone or more annotated source images, by electronically insertingelements/features of interest and/or background elements/features intothe copies, by electronically varying modality-based characteristics ofthe copies, and/or by electronically applying geometric transformationsto the copies. The resulting plurality of training images are a highlyvaried set of images that approximate and/or simulate real-worldvariability (e.g., element/feature insertion can help to approximatereal-world biological variability; modality-based characteristicvariation can help to approximate real-world device modalityvariability; and geometric characteristic variation can help to furtherapproximate real-world variability). Training a machine learning modelon such a plurality of training images can result in significantlyimproved performance and/or efficacy as compared to training a machinelearning model only on the one or more annotated source images. So, sucha computerized system is clearly a useful and practical application ofcomputers.

Moreover, various embodiments of the invention can provide technicalimprovements to and solve problems that arise in the field of trainingof machine learning models. As explained above, the efficacy and/orperformance of a machine learning model can be limited by the veracity,volume, variety, and/or velocity of training data (e.g., training datathat inadequately simulates real-world variability can result ininadequate machine learning models). Embodiments of the subjectinnovation address this technical problem by providing a computerizedsystem that can quickly synthetically generate veracious, voluminous,and/or varied training data (e.g., element/feature insertion,modality-based characteristic variation, and geometric transformationscan all help to simulate and/or approximate real-world variability).Training a machine learning model on such veracious, voluminous, and/orvaried training data can result in significantly improved modelperformance. Because embodiments of the subject innovation can improvethe very computing performance of machine learning models, embodimentsof the subject innovation constitute a technical improvement.

Furthermore, various embodiments of the subject innovation can controlreal-world devices based on the disclosed teachings. For example,embodiments of the subject innovation can electronically receive areal-world annotated source image (e.g., X-ray scan, CT scan, MRI scan,PET scan, ultrasound scan, visible-light-spectrum photograph).Embodiments of the subject innovation can electronically insertreal-world images of elements/features of interest and/or real-worldimages of background elements/features into the real-world annotatedsource image. Embodiments of the subject innovation can electronicallyvary real-world modality-based characteristics of the real-worldannotated source image. Moreover, embodiments of the subject innovationcan electronically vary real-world geometric characteristics of thereal-world annotated source image. Such electronic insertions and/orelectronic variations can result in a plurality of real-world trainingimages that more fully and/or more completely simulate real-world imagevariability. Training a real-world machine learning model on such aplurality of real-world training images can result in enhancedefficacy/performance of the machine learning model, which is a concreteand tangible technical improvement.

It should be appreciated that the herein figures are exemplary andnon-limiting.

FIG. 1 illustrates a block diagram of an example, non-limiting system100 that can facilitate synthetic training data generation for improvedmachine learning model generalizability in accordance with one or moreembodiments described herein. As shown, it can be desired to train amachine learning model 106 on an annotated source image 104. However,the annotated source image 104 can, in various aspects, fail to fullyand/or adequately represent the full space of real-world imagevariability. In various instances, a synthetic training data generationsystem 102 can address this problem by electronically generating a setof training images based on the annotated source image 104 and on whichthe machine learning model 106 can be trained.

In various aspects, the machine learning model 106 can be any suitablecomputationally-implemented artificial intelligence model and/oralgorithm that is designed to receive as input one or more images and toproduce as output one or more classifications, labels, and/orpredictions based on the inputted one or more images (e.g., supportvector machine, neural network, expert system, Bayesian belief network,fuzzy logic, data fusion engine, and/or so on). In various aspects, anysuitable machine learning model and/or algorithm can be implemented,such as a model and/or algorithm for performing classifications, forperforming segmentations, for performing detections, for performingregressions, for performing reconstructions, for performingimage-to-image (and/or data-to-data) transformations, and/or forperforming any other suitable machine learning functionality.

In various aspects, the annotated source image 104 can be any suitableimage which the machine learning model 106 is designed to analyze. Forexample, if the machine learning model 106 is designed to classifymedical images, the annotated source image 104 can be any suitablemedical image (e.g., X-ray scan of a patient, CT scan of a patient, MRIscan of a patient, PET scan of a patient, ultrasound scan of a patient,visible-light-spectrum photograph of a patient). As explained above, theannotated source image 104 can have a corresponding and/or associatedannotation (e.g., a classification and/or label that is considered aground truth for the annotated source image 104).

In various embodiments, the synthetic training data generation system102 can electronically receive/retrieve (e.g., via any suitable wiredand/or wireless electronic connection) the annotated source image 104.In various aspects, the synthetic training data generation system 102can electronically receive/retrieve the annotated source image 104 fromany suitable database and/or data structure that is accessible to thesynthetic training data generation system 102. In various aspects, thesynthetic training data generation system 102 can electronicallyreceive/retrieve the annotated source image 104 directly from animage-capture device that generates, captures, and/or creates theannotated source image 104 (e.g., directly from an X-ray scanner, from aCT scanner, from a PET scanner, from an MRI scanner)

In various embodiments, the synthetic training data generation system102 can comprise a processor 108 (e.g., computer processing unit,microprocessor) and a computer-readable memory 110 that is operablyand/or operatively and/or communicatively connected/coupled to theprocessor 108. The memory 110 can store computer-executable instructionswhich, upon execution by the processor 108, can cause the processor 108and/or other components of the synthetic training data generation system102 (e.g., element augmentation component 112, modality augmentationcomponent 114, geometry augmentation component 116, training component118) to perform one or more acts. In various embodiments, the memory 110can store computer-executable components (e.g., element augmentationcomponent 112, modality augmentation component 114, geometryaugmentation component 116, training component 118), and the processor108 can execute the computer-executable components.

In various embodiments, the synthetic training data generation system102 can comprise an element augmentation component 112. In variousaspects, the element augmentation component 112 can generate a set ofpreliminary training images based on the annotated source image 104.Specifically, the element augmentation component 112 can comprise anelement catalog that electronically stores images of elements (e.g.,elements of interest and/or background elements) that are insertableinto the annotated source image 104 (e.g., insertable into copies of theannotated source image 104). In various aspects, the elementaugmentation component 112 can form/generate a preliminary trainingimage by making an electronic copy of the annotated source image 104 andby inserting at least one element from the element catalog into theelectronic copy of the annotated source image 104.

It should be appreciated that when the herein disclosure discussesinserting elements into the annotated source image 104, this can includeinserting elements into one or more copies of the annotated source image104.

In various instances, the elements that are stored within the elementcatalog can depend upon the nature of the machine learning model 106.For example, the element catalog can include images of elements ofinterest and can include images of background elements. In variousaspects, an element of interest can be any suitable visual object thatthe machine learning model 106 is supposed to learn, predict, detect,and/or classify. In various cases, a background element can be anysuitable visual object that the machine learning model 106 need notlearn, predict, detect, and/or classify, but which can impede and/orthrow off the machine learning model 106. For example, if the machinelearning model 106 is configured to learn, predict, detect, and/orclassify lung growths, elements of interest can include variousmalignant lung growths and/or various benign lung growths, andbackground elements can include various medical equipment (e.g.,pacemaker, intravenous tubing, stents, implants, electrocardiogramleads), various co-morbidities (e.g., heart defects, occluded bloodvessels), stomach gas, and/or so on. In other words, in the medicalcontext, an element of interest can be any suitable anatomical structureand/or biological symptom manifestation that the machine learning model106 is supposed to learn, predict, and/or detect, and a backgroundelement can be any other suitable anatomical structure and/or biologicalsymptom manifestation which can distract and/or impede the machinelearning model 106 and/or can be any suitable piece of medical equipmentwhich can distract and/or impede the machine learning model 106.

In various aspects, the element augmentation component 112 can insertany suitable combination of elements from the element catalog into theannotated source image 104 to create a preliminary training image (e.g.,each preliminary training image can have one inserted element, eachpreliminary training image can have multiple inserted elements,different preliminary training images can have different numbers ofinserted elements, and/or at least one preliminary training image canhave no inserted elements).

In various aspects, the element augmentation component 112 can localizean inserted element in the annotated source image 104 in any suitable,biologically-possible location/position. For instance, if an image of alung lesion is inserted by the element augmentation component 112, theimage of the lung lesion can be placed in a depicted chest cavity of theannotated source image 104 and can avoid being placed in a depictedabdominal cavity of the annotated source image 104 (e.g., lung lesionscan possibly form in the chest cavity but cannot possibly form in theabdominal cavity). In this way, a same element can be inputted intodifferent locations/positions of the annotated source image 104, therebyyielding different preliminary training images.

In various instances, the element augmentation component 112 can controlthe orientation of an inserted element in the annotated source image104. For example, if an image of a lung lesion is inserted by theelement augmentation component 112, the image can be oriented asdepicted in the element catalog, can be oriented upside-down, can beoriented backwards, can be oriented sideways, can be reflected/rotatedin any suitable manner, and/or so on. In this way, a same element can bedifferently oriented in a same location of the annotated source image104, thereby yielding different preliminary training images.

In some cases, the element augmentation component 112 can controldimensions and/or intensities of an inserted element in the annotatedsource image 104. For example, if an image of a lung lesion if insertedby the element augmentation component 112, the image of the lung lesioncan be expanded, contracted, lengthened, widened, thickened, manipulatedin any other suitable way, and/or so on. In this way, a same element canbe differently sized in a same location and/or same orientation of theannotated source image 104, thereby yielding different preliminarytraining images.

In various instances, when a preliminary training image is formed byinserting only background elements into the annotated source image 104,an annotation of the preliminary training image can be the same as theannotation of the annotated source image 104 (e.g., if an X-ray image isannotated as depicting one type of lung cancer, adding stomach gas tothat X-ray image can fail to affect the accuracy/completeness of theannotation). In various aspects, when a preliminary training image isformed by inserting an element of interest into the annotated sourceimage 104, an annotation of the preliminary training image can beinitialized as the annotation of the annotated source image 104 and canthen be updated based on the inserted element of interest (e.g., if anX-ray image is annotated as depicting one type of lung cancer, adding asecond type of lung cancer to that X-ray image can affect theaccuracy/completeness of the annotation; accordingly, the annotation canbe updated to indicate that the X-ray image now depicts two types oflung cancers).

In various embodiments, the synthetic training data generation system102 can comprise a modality augmentation component 114. In variousaspects, the modality augmentation component 114 can generate a set ofintermediate training images based on the set of preliminary trainingimages generated by the element augmentation component 112.Specifically, the modality augmentation component 114 can comprise alist of various modality-based characteristics. In various aspects, amodality-based characteristic can be any suitable image property that isrelated to and/or dependent upon a device modality that captured and/orgenerated the annotated source image 104. For example, modality-basedcharacteristics can include gamma/radiation levels (e.g., sincegamma/radiation is used to generate X-rays and/or CT scans), brightnesslevels, contrast levels, blur levels, noise levels, image texture, imagefield of view, image resolution, and/or image artifacts (e.g., glare onlens, scratch on lens, dust on lens). In other words, modality-basedcharacteristics can represent parameters of image-capture devices, whichparameters can affect the quality/properties of the captured images. Invarious cases, the modality augmentation component 114 can form/generatean intermediate training image by making an electronic copy of apreliminary training image and by varying/adjusting at least onemodality-based characteristic of the preliminary training image.

It should be appreciated that when the herein disclosure discussesvarying modality-based characteristics of a preliminary training image,this can include varying modality-based characteristics of an electroniccopy of the preliminary training image.

In various aspects, the modality augmentation component 114 canvary/adjust/manipulate/modify any suitable combination of modality-basedcharacteristics of a preliminary training image to create anintermediate training image (e.g., each intermediate training image canbe formed by varying one modality-based characteristic, eachintermediate training image can be formed by varying multiplemodality-based characteristics, different intermediate training imagescan be formed by varying different numbers of modality-basedcharacteristics, and/or at least one intermediate training image caninvolve no variation of any modality-based characteristics).

In various instances, varying and/or modifying modality-basedcharacteristics can have no effect on the accuracy and/or completenessof annotations. Accordingly, an intermediate training image that isformed from a preliminary training image can have the same annotation asthe preliminary training image.

In various embodiments, the synthetic training data generation system102 can comprise a geometry augmentation component 116. In variousaspects, the geometry augmentation component 116 can generate a set ofdeployable training images based on the set of intermediate trainingimages generated by the modality augmentation component 114.Specifically, the geometry augmentation component 116 can comprise alist of various geometric transformations that can be applied to animage. In various aspects, a geometric transformation can be anysuitable mathematical operation that can transform spatial properties ofan image. For example, geometric transformations can include reflectionsof an image about any suitable axis, rotations of an image about anysuitable axis, panning and/or tilting an image to change atwo-dimensional projection and/or perspective of the image, cropping anysuitable portion of an image, zooming in and/or out on an image,optically distorting an image (e.g., barrel distortion, pincushiondistortion, mustache distortion, and/or any other suitable distortionaway from a rectilinear projection), and/or so on. In various cases, thegeometry augmentation component 116 can form/generate a deployabletraining image by making an electronic copy of an intermediate trainingimage and by applying at least one geometric transformation to theelectronic copy of the intermediate training image.

It should be appreciated that when the herein disclosure discussesapplying geometric transformations to an intermediate training image,this can include applying geometric transformations to an electroniccopy of the intermediate training image.

In various aspects, the geometry augmentation component 116 can applyany suitable combination of geometric transformations to an intermediatetraining image to create a deployable training image (e.g., eachdeployable training image can be formed by applying one geometrictransformation, each deployable training image can be formed by applyingmultiple geometric transformations, different deployable training imagescan be formed by applying different numbers of geometrictransformations, and/or at least one deployable training image canundergo no geometric transformations).

In various instances, applying geometric transformations can have noeffect on the accuracy and/or completeness of annotations. Accordingly,a deployable training image that is formed from an intermediate trainingimage can have the same annotation as the intermediate training image.

In various embodiments, the synthetic training data generation system102 can comprise a training component 118. In various aspects, thetraining component 118 can actually train (e.g., via backpropagationand/or any other suitable technique) the machine learning model 106 onthe set of deployable training images generated by the synthetictraining data generation system 102.

FIG. 2 illustrates a block diagram of an example, non-limiting system200 including an element catalog that can facilitate synthetic trainingdata generation for improved machine learning model generalizability inaccordance with one or more embodiments described herein. As shown, thesystem 200 can, in some cases, comprise the same components as thesystem 100, and can further comprise an element catalog 202 andpreliminary training images 204.

In various aspects, the element augmentation component 112 can comprisethe element catalog 202. In various instances, the element augmentationcomponent 112 can leverage the element catalog 202 to generatepreliminary training images 204 based on the annotated source image 104.As mentioned above, the element catalog 202 can electronically storeand/or maintain images of elements/features that are insertable into theannotated source image 104. Specifically, the element catalog 202 caninclude elements of interest and/or background elements. In variouscases, an element interest can be any suitable visual object that themachine learning model 106 is supposed to learn, predict, and/or detect(e.g., if the machine learning model 106 is configured to detectoccluded blood vessels in a patient's brain, elements of interest can bevarious images of occluded blood vessels). In various aspects, abackground element can be any suitable visual object that can impedeand/or distract the machine learning model 106 (e.g., if the machinelearning model 106 is configured to detect occluded blood vessels in apatient's brain, background elements can be various images of brainlesions and/or various images of cranial and/or cerebral implants).

In various instances, the elements stored in the element catalog 202 canbe generated via any suitable technique and/or can be stored in anysuitable computerized format. In some cases, elements stored within theelement catalog 202 can be scanned images of hand-drawn figures (e.g., amedical professional can sketch a brain lesion, a lung nodule, and/orintravenous tubing by hand, and the sketch and can be scanned and savedelectronically within the element catalog 202). In some cases, elementsstored within the element catalog 202 can be two-dimensional and/orthree-dimensional computer-aided-design models (e.g., a medicalprofessional can generate on a computer a two-dimensional and/orthree-dimensional computer-aided-design model of a brain lesion, a lungnodule, and/or intravenous tubing, and the two-dimensional and/orthree-dimensional computer-aided-design model can be saved and/or storedelectronically within the element catalog 202). In various aspects, anyother suitable technique can be implemented to generate and/or obtainthe elements within the element catalog 202 (e.g., elements within theelement catalog 202 can be cut-outs from existing images, and/or so on).

In various aspects, as mentioned above, the element augmentationcomponent 112 can control and/or manipulate any suitable visualcharacteristics of the elements within the element catalog 202. Forexample, the element augmentation component 112 can change/modify anysuitable spatial dimensions of the elements in the element catalog 202(e.g., length, width, height, thickness, color, shading, intensity,and/or so on). As another example, the element augmentation component112 can change/modify depicted and/or projected orientations of theelements in the element catalog 202 (e.g., can depict the elementsfacing forward, facing backward, facing upside down, facing sideways,rotated by any suitable magnitude about any suitable axis, reflectedabout any suitable axis, and/or so on). In various aspects,modifying/changing dimensions and/or orientations can be morerealistically and/or more fully facilitated if computer-aided-designmodels are implemented, as mentioned above.

In various cases, as explained above, the element augmentation component112 can localize an inserted element within the annotated source image104 in any suitable, biologically-possible location (e.g., stomach gascan be inserted into any portion of a depicted abdominal cavity, butcannot be inserted into any portion of a depicted chest cavity). Thus,in various aspects, the element catalog 202 can map and/or correlatedifferent elements with different biologically-possible locations, andthe element augmentation component 112 can localization elements duringinsertion based on the mapping and/or correlation.

In various aspects, the element augmentation component 112 can insertelements into the annotated source image 104 according to any suitableaugmentation policy and/or scheme.

In various instances, the element catalog 202 can be considered as aparametrization of the space of possible/potential elements/featuresthat are insertable into the annotated source image 104. In other words,a space of all possible/potential image elements/features which can bedepicted in the annotated source image 104 can be conceived, and theelement catalog 202 can be constructed and/or configured so as to spanand/or substantially span that space. In various cases, such aparametrized space can depend upon the operational context of themachine learning model 106 (e.g., in the medical context, the space cancomprise possible/potential biological symptom manifestations that canbe captured in an image and/or possible/potential medical equipment thatcan be captured in an image).

In various embodiments, the element augmentation component 112 canupdate and/or change the element catalog 202 (e.g., can update and/orchange the images listed/stored in the element catalog 202 that are usedto generate the preliminary training images 204). For instance, in somecases, the element catalog 202 can be initialized with an existing setof images of elements of interest and/or an existing set of images ofbackground elements. However, in various aspects, the elementaugmentation component 112 can periodically and/or aperiodically queryany suitable database and/or data structure which is accessible to theelement augmentation component 112 to check if new images of elements ofinterest and/or new images of background elements are available (e.g.,to check if images that are not already stored/listed within the elementcatalog 202 are available for retrieval and/or download so that such newimages can be used to generate the preliminary training images 204). Ifsuch new images of elements of interest and/or background elements areavailable in the database and/or data structure, the elementaugmentation component 112 can retrieve such new images and add them tothe element catalog 202 and can thus begin inserting such new imagesinto the annotated source image 104 to generate the preliminary trainingimages 204. As another example, the element augmentation component 112can receive input from an operator, which input includes a new image ofan element of interest and/or background element that is not alreadystored/listed in the element catalog 202. In various aspects, theelement augmentation component 112 can accordingly add the new image tothe element catalog 202 and can thus begin using the new image togenerate the preliminary training images 204. In this way, the elementcatalog 202 can be updated, changed, amended, edited, and/or modified asdesired so as to suit different operational contexts.

As an example, suppose that the element catalog 202 includes an image ofa lung nodule, an image of stomach gas, and an image of breathing tubes.Thus, the element augmentation component 112 can insert into theannotated source image 104 different combinations/permutations of theimage of the lung nodule, the image of stomach gas, and the image ofbreathing tubes (e.g., with different localizations and/or orientationsand/or dimensions) to generate the preliminary training images 204. Invarious aspects, the element augmentation component 112 can retrievefrom any suitable database and/or data structure (and/or can receive asinput from an operator) an image of a pacemaker. Since the image of thepacemaker is not already stored/listed within the element catalog 202,the element augmentation component 112 can add the image of thepacemaker to the element catalog 202. Thus, the element augmentationcomponent 112 can begin inserting the image of the pacemaker (e.g., withdifferent localizations and/or different orientations and/or differentdimensions) into the annotated source image 104 to generate thepreliminary training images 204. In this way, the element catalog 202can be updated and/or enlarged over time and/or as desired.

FIGS. 3-4 illustrate block diagrams of example, non-limiting preliminarytraining images 300 and 400 formed from an annotated source image inaccordance with one or more embodiments described herein.

As shown in FIG. 3 , the preliminary training images 204 can begenerated from the annotated source image 104. In various cases, therecan be N preliminary training images 204, for any suitable integer N. Inother words, the element augmentation component 112 can create Nelectronic copies of the annotated source image 104, and can insert anysuitable number and/or combinations/permutations of elements from theelement catalog 202 into each of the N electronic copies of theannotated source image 104, thereby generating the N preliminarytraining images 204. As explained above, a goal of element insertion canbe to increase the feature variety and/or diversity that is depicted inthe annotated source image 104. Accordingly, the element augmentationcomponent 112 can insert different numbers of different elements havingdifferent orientations and/or different dimensions into differentlocations of different copies of the annotated source image 104, therebygenerating the preliminary training images 204. In other words, thesingle annotated source image 104 can be converted into the Npreliminary training images 204.

As mentioned above, if a particular preliminary training image is formedby inserting only background elements or by inserting no elements atall, the annotation of the particular preliminary training image can bethe same as the annotation of the annotated source image 104 (e.g.,background elements can have no effect on the accuracy and/orcompleteness of an annotation). However, if a particular preliminarytraining image is formed by inserting any elements of interest, theannotation of the particular preliminary training image can beinitialized as the the annotation of the annotated source image 104 andcan be adjusted to reflect the inserted elements of interest. In thisway, all of the preliminary training images 204 can have annotationsbased on the annotation of the annotated source image 104 and/or basedon the inserted elements.

FIG. 4 depicts a real-world example showing how the annotated sourceimage 104 can be used to create the preliminary training images 204. Asshown, there can be an initial chest X-ray 402. In various cases, theinitial chest X-ray 402 can be considered as the annotated source image104. In various aspects, varied chest X-rays 404 can be generated byinserting various elements into the initial chest X-ray 402. AlthoughFIG. 4 depicts sixteen varied chest X-rays 404 arranged in afour-by-four grid, this is exemplary and non-limiting. For ease ofexplanation, suppose that the top-most row is row 1 and the bottom-mostrow is row 4, and suppose that the left-most column is column 1 and theright-most column is column 4. As shown, the image at (row 1, column 1),the image at (row 1, column 4), the image at (row 2, column 3), and theimage at (row 4, column 4) of the varied chest X-rays 404 can be formedby inserting and/or superimposing various features (e.g., stomach gas,intestinal growths/cists, colon cancers, digestive dyes, and/or so on)into and/or on the depicted abdominal cavity of the initial chest X-ray402. As shown, the image at (row 1, column 3), the image at (row 2,column 1), the image at (row 2, column 2), the image at (row 2, column4), the image at (row 3, column 4), and the image at (row 4, column 1)of the varied chest X-rays 404 can be formed by inserting and/orsuperimposing various features (e.g., intravenous tubing, breathingtubes, electrocardiogram wires/leads, pacemakers, and/or so on) intoand/or on the depicted chest cavity of the initial chest X-ray 402. Asshown, the image at (row 1, column 2), the image at (row 3, column 1),the image at (row 3, column 2), the image at (row 3, column 3), and theimage at (row 4, column 2) of the varied chest X-rays 404 can be formedby inserting and/or superimposing various features (e.g., chestgrowths/nodules/masses, fluid-filled sacs, lacuna, consolidation, and/orso on) into and/or on the depicted chest cavity of the initial chestX-ray 402. Lastly, as shown, the image at (row 4, column 3) of thevaried chest X-rays 404 can be formed by inserting and/or superimposingvarious features (e.g., metal screws, rods, and/or implants) into and/oron the initial chest X-ray 402.

Overall, different elements having different dimensions can bedifferently oriented in different locations of the initial chest X-ray402 in order to create the varied chest X-rays 404.

FIG. 5 illustrates a block diagram of an example, non-limiting system500 including modality-based characteristics that can facilitatesynthetic training data generation for improved machine learning modelgeneralizability in accordance with one or more embodiments describedherein. As shown, the system 500 can, in some cases, comprise the samecomponents as the system 200, and can further comprise modality-basedcharacteristics 502 and intermediate training images 504.

In various aspects, the modality augmentation component 114 can comprisea list of modality-based characteristics 502 that are applicable to thepreliminary training images 204. In various instances, the modalityaugmentation component 114 can vary, change, and/or modify any of themodality-based characteristics 502 of the preliminary training images204 to generate intermediate training images 504. As mentioned above,the modality-based characteristics 502 can include any suitable imageproperty that depends upon and/or is related to the image-capture devicethat generated the annotated source image 104. For example, themodality-based characteristics 502 can include gamma/radiation levelsexhibited by and/or depicted in an image, brightness levels exhibited byand/or depicted in an image, contrast levels exhibited by and/ordepicted in an image, blur levels exhibited by and/or depicted in animage, noise levels exhibited by and/or depicted in an image, textureexhibited by and/or depicted in an image, field of view exhibited byand/or depicted in an image, resolution exhibited by and/or depicted inan image, device artifacts (e.g., lens scratches, lens dust, lensglares) exhibited by and/or depicted in an image, and/or so on. Invarious aspects, the modality augmentation component 114 can generatethe intermediate training images 504 by varying, changing, and/ormodifying any of the modality-based characteristics 502 of thepreliminary training images 204 (e.g., different electronic copies ofeach of the preliminary training images 204 can be made, and differentmodality-based characteristics (e.g., 502) of the different electroniccopies can be differently varied to generate the intermediate trainingimages 504).

In various instances, the list of modality-based characteristics 502 canbe considered as a parametrization of the space of possible/potentialimage properties that depend upon the device modality that generatedand/or captured the annotated source image 104. In other words, a spaceof possible/potential image properties which can vary from image-capturedevice modality to image-capture device modality can be conceived, andthe list of modality-based characteristics 502 can be constructed and/orconfigured so as to span and/or substantially span that space. Invarious cases, such a parametrized space can depend upon the operationalcontext of the machine learning model 106.

In various embodiments, the modality augmentation component 114 canupdate and/or change the list of modality-based characteristics 502(e.g., can update and/or change the list of modifiable imagecharacteristics/properties that are related to and/or associated withdevice modality and that are used to generate the intermediate trainingimages 504). For instance, in some cases, the list of modality-basedcharacteristics 502 can be initialized with an existing set ofmodifiable image characteristics/properties that are related to devicemodality. However, in various aspects, the modality augmentationcomponent 114 can periodically and/or aperiodically query any suitabledatabase and/or data structure which is accessible to the modalityaugmentation component 114 to check if new modifiable imagecharacteristics/properties related to device modality are available(e.g., to check if image characteristics/properties that depend upondevice modality and that are not currently flagged/marked as modifiableare known so that such new image characteristics/properties can be usedto generate the intermediate training images 504). If it is determinedthat such new modifiable image characteristics/properties related todevice modality are available, the modality augmentation component 114can include such new image characteristics/properties in the list ofmodality-based characteristics 502 and can thus begin modifying such newimage characteristics/properties when generating the intermediatetraining images 504. As another example, the modality augmentationcomponent 114 can receive input from an operator, which input indicatesa new image characteristic/property that is not already included in thelist of modality-based characteristics 502. In various aspects, themodality augmentation component 114 can accordingly add the new imagecharacteristic/property to the list of modality-based characteristics502 and can thus begin modifying the new characteristic/property togenerate the intermediate training images 504. In this way, the list ofmodality-based characteristics 502 can be updated, changed, amended,edited, and/or modified as desired so as to suit different operationalcontexts.

As an example, suppose that the list of modality-based characteristics502 includes image gamma/radiation level, image brightness level, andimage contrast level. Thus, the modality augmentation component 114 canmodify and/or vary different combinations/permutations ofgamma/radiation level, brightness level, and/or contrast level of thepreliminary training images 204 in order to generate the intermediatetraining images 504. In various aspects, the modality augmentationcomponent 114 can retrieve from any suitable database and/or datastructure (and/or can receive as input from an operator) an indicationthat image blur level is now a modifiable image property that is relatedto device modality. Since the list of modality-based characteristics 502does not already include image blur level, the modality augmentationcomponent 114 can add image blur level to the list of modality-basedcharacteristics 502. Thus, the modality augmentation component 114 canbegin altering/modifying image blur level of the preliminary trainingimages 204 in order to generate the intermediate training images 504. Inthis way, the list of modality-based characteristics 502 can be updatedand/or enlarged over time and/or as desired.

FIG. 6 illustrates a block diagram of example, non-limiting intermediatetraining images 600 formed from preliminary training images inaccordance with one or more embodiments described herein.

As shown in FIG. 6 , the intermediate training images 504 can begenerated from the preliminary training images 204. In various cases,there can be M intermediate training images 504 for each of thepreliminary training images 204, for any suitable integer M (e.g.,intermediate training image 1_1 to intermediate training image 1_Mformed from the preliminary training image 1; intermediate trainingimage N_1 to intermediate training image N_M formed from the preliminarytraining image N, and/or so on). In other words, the modalityaugmentation component 114 can create M electronic copies of each of theN preliminary training images 204, and can vary, change, and/or modifyany suitable number of modality-based characteristics (e.g., 502) ofeach of the M electronic copies of each of the N preliminary trainingimages 204, thereby generating a total of N*M intermediate trainingimages 504. As explained above, a goal of modality-based characteristicmodification can be to increase the variety and/or diversity that isdepicted in the preliminary training images 204. Accordingly, themodality augmentation component 114 can differently adjust differentcombinations/permutations of different modality-based characteristics ofdifferent copies of the preliminary training images 204, therebygenerating the intermediate training images 504. In other words, thesingle annotated source image 104 can be converted into the N*Mintermediate training images 504.

As mentioned above, modification of any of the modality-basedcharacteristics 502 can have no effect on the accuracy and/orcompleteness of an annotation. Accordingly, a particular intermediatetraining image formed from a particular preliminary training image canhave the same as annotation as the particular preliminary trainingimage.

FIG. 7 illustrates a block diagram of an example, non-limiting system700 including geometric transformations that can facilitate synthetictraining data generation for improved machine learning modelgeneralizability in accordance with one or more embodiments describedherein. As shown, the system 700 can, in various instances, comprise thesame components as the system 500, and can further comprise geometrictransformations 702 and deployable training images 704.

In various aspects, the geometry augmentation component 116 can comprisea list of geometric transformations 702 that are applicable to theintermediate training images 504. In various instances, the geometryaugmentation component 116 can apply any of the geometrictransformations 702 to the intermediate training images 504 to generatedeployable training images 704. As mentioned above, the geometrictransformations 702 can include any suitable mathematic transformationand/or operation that can spatially alter the depicted geometry of animage (e.g., of the intermediate training images 504). For example, thelist of geometric transformations 702 can include reflecting an imageabout any suitable axis, rotating an image by any suitable magnitudeabout any suitable axis, panning an image in any suitable direction byany suitable magnitude, tilting an image in any suitable direction byany suitable magnitude, zooming in and/or out on any suitable portion ofan image by any suitable magnitude, cropping any suitable portion of animage in any suitable way, expanding an image in any suitable directionand by any suitable magnitude, contracting an image in any suitabledirection and by any suitable magnitude, distorting any suitable portionof an image in any suitable way and by any suitable magnitude,harmonizing and/or de-harmonizing an image in any suitable way and byany suitable magnitude, applying any suitable affine and/or elastictransformation to an image in any suitable way, and/or so on. In variousaspects, the geometry augmentation component 116 can generate thedeployable training images 704 by applying any of the geometrictransformations 702 to the intermediate training images 504 (e.g.,different electronic copies of each of the intermediate training images504 can be made, and different geometric transformations (e.g., 702) ofthe different electronic copies can be applied to generate thedeployable training images 704).

In various instances, the geometric transformations 702 can beconsidered as a parametrization of the space of possible/potentialmathematical transformations that can be applied to an image. In otherwords, a space of possible/potential mathematical transformations and/oroperations which can be applied to an image can be conceived, and thelist of geometric transformations 702 can be constructed and/orconfigured so as to span and/or substantially span that space. Invarious cases, such a parametrized space can depend upon the operationalcontext of the machine learning model 106.

In various embodiments, the geometry augmentation component 116 canupdate and/or change the list of geometric transformations 702 (e.g.,can update and/or change the list of mathematicaloperations/transformations that are used to generate the deployabletraining images 704). For instance, in some cases, the list of geometrictransformations 702 can be initialized with an existing set ofmathematical operations/transformations that are appliable to images.However, in various aspects, the geometry augmentation component 116 canperiodically and/or aperiodically query any suitable database and/ordata structure which is accessible to the geometry augmentationcomponent 116 to check if new mathematical operations/transformationsappliable to images are available (e.g., to check if mathematicaloperations/transformations that are not currently flagged/marked asappliable to images are known so that such new mathematicaloperations/transformations can be used to generate the deployabletraining images 704). If it is determined that such new mathematicaloperations/transformations are available, the geometry augmentationcomponent 116 can include such new mathematicaloperations/transformations in the list of geometric transformations 702and can thus begin applying such new mathematicaloperations/transformations when generating the deployable trainingimages 704. As another example, the geometry augmentation component 116can receive input from an operator, which input indicates a newmathematical operation/transformation that is not already included inthe list of geometric transformations 702. In various aspects, thegeometry augmentation component 116 can accordingly add the newmathematical operation/transformation to the list of geometrictransformations 702 and can thus begin applying the new mathematicaloperation/transformation to generate the deployable training images 704.In this way, the list of geometric transformations 702 can be updated,changed, amended, edited, and/or modified as desired so as to suitdifferent operational contexts.

As an example, suppose that the list of geometric transformations 702includes image rotating, image reflecting, and image tilting. Thus, thegeometry augmentation component 116 can apply differentcombinations/permutations of image rotations, image reflections, and/orimage tilts to the intermediate training images 504 in order to generatethe deployable training images 704. In various aspects, the geometryaugmentation component 116 can retrieve from any suitable databaseand/or data structure (and/or can receive as input from an operator) anindication that image distorting is now an available geometrictransformation. Since the list of geometric transformations 702 does notalready include image distorting, the geometry augmentation component116 can add image distorting to the list of geometric transformations702. Thus, the geometry augmentation component 116 can begin applyingimage distortion to the intermediate training images 504 in order togenerate the deployable training images 704. In this way, the list ofgeometric transformations 702 can be updated and/or enlarged over timeand/or as desired.

FIG. 8 illustrates a block diagram of example, non-limiting deployabletraining images 800 formed from intermediate training images inaccordance with one or more embodiments described herein.

As shown in FIG. 8 , the deployable training images 704 can be generatedfrom the intermediate training images 504. In various cases, there canbe P deployable training images 704 for each of the intermediatetraining images 504, for any suitable integer P (e.g., deployabletraining image 1_1_1 to deployable training image 1_1_P formed fromintermediate training image 1_1; deployable training image N_M_1 todeployable training image N_M_P formed from intermediate training imageN_M; and/or so on). In other words, the geometry augmentation component116 can create P electronic copies of each of the N*M intermediatetraining images 504, and can apply any suitable number of geometrictransformations (e.g., 702) to each of the P electronic copies of eachof the N*M intermediate training images 504, thereby generating a totalof N*M*P deployable training images 704. As explained above, a goal ofgeometric transformation can be to increase the variety and/or diversitythat is depicted in the intermediate training images 504. Accordingly,the geometry augmentation component 116 can apply differentcombinations/permutations of different geometric transformations todifferent copies of the intermediate training images 504, therebygenerating the deployable training images 704. In other words, thesingle annotated source image 104 can be converted into the N*M*Pdeployable training images 704.

As mentioned above, application of any of the geometric transformations702 can have no effect on the accuracy and/or completeness of anannotation. Accordingly, a particular deployable training image formedfrom a particular intermediate training image can have the sameannotation as the particular intermediate training image.

As shown, in various aspects, the number of deployable training images704 can be greater than the number of intermediate training images 504,which can be greater than the number of preliminary training images 204.

FIG. 9 illustrates a block diagram of example, non-limiting variationsof modality-based characteristics and/or geometric characteristics inaccordance with one or more embodiments described herein.

In other words, FIG. 9 depicts a real-world example showing how thepreliminary training images 204 can be used to create the intermediatetraining images 504 and/or the deployable training images 704. As shown,there can be augmented X-rays 902. In various cases, modality-basedcharacteristics of the augmented X-rays 902 and/or geometriccharacteristics of the augmented X-rays 902 can be manipulated and/ormodified as described above to create the further-augmented X-rays 904.Although FIG. 9 depicts sixteen further-augmented X-rays 904 arranged ina four-by-four grid, this is exemplary and non-limiting. For ease ofexplanation, suppose that the top-most row is row 1 and the bottom-mostrow is row 4, and suppose that the left-most column is column 1 and theright-most column is column 4. As shown, the image at (row 1, column 3),the image at (row 2, column 2), the image at (row 3, column 2), theimage at (row 4, column 1), and the image at (row 4, column 3) of thefurther augmented X-rays 904 can be formed by increasing and/ordecreasing brightness/contrast/gamma levels of the augmented X-rays 902.As shown, the image at (row 1, column 2), the image at (row 2, column1), the image at (row 2, column 3), the image at (row 2, column 4), theimage at (row 3, column 2), the image at (row 3, column 3), the image a(row 4, column 1), the image at (row 4, column 2), the image at (row 4,column 3), and the image at (row 4, column 4) of the further-augmentedX-rays 904 can be formed by cropping, zooming, and/or opticallydistorting the augmented X-rays 902. In various cases, any othersuitable transformations and/or modifications are possible.

In various aspects, as mentioned above, the training component 218 cantrain the machine learning model 106 on the deployable training images704.

FIG. 10 illustrates example, non-limiting experimental results 1000 inaccordance with one or more embodiments described herein.

In various aspects, the inventors of various embodiments of the subjectinnovation generated training data sets as described herein (e.g., viaelement/feature insertion, via modality-based characteristicmodification, via geometric transformation), and their experimentsrevealed that a lung-segmentation machine learning model (e.g., 106)trained on the generated training data sets exhibited significantlyimproved performance/efficacy as compared to being trained on aconventional training data set. Specifically, as shown in FIG. 10 , fourdifferent trials were conducted in which it was desired to train thelung-segmentation machine learning model on an original data set of size138 (e.g., 138 training images in the original data set). A first trialwas performed in which the machine learning model was trained only onthe original data set. A second trial was performed in whichelement/feature insertion was performed to a limited extent (e.g.,denoted by the small green annular ring). In this second trial, theelement/feature insertion caused the original data set to grow from asize of 138 to a size of 1600 (e.g., after element/feature insertion,the data set included 1600 training images). A third trial was performedin which element/feature insertion was performed to a greater extent(e.g., denoted by the large green annular ring). In this third trial,the element/feature insertion caused the original data set to grow froma size of 138 to a size of 3670. Lastly, a fourth trial was performed inwhich element/feature insertion was performed to the greater extent andin which modality-based characteristics were varied and geometrictransformations were applied (e.g., denoted by the blue annular ring).In this fourth trial, the element/feature insertion, the modality-basedcharacteristic variation, and the geometric transformations caused theoriginal data set to grow from a size of 138 to a size of 73,640.

Various performance metrics of the trained machine learning model foreach of these four trials are depicted in FIG. 10 . The inventors used atest data set of size 1966 to test the performance/efficacy of thetrained machine learning model in each trial. As shown, in the firsttrial (e.g., trained only on the original data size), the machinelearning model achieved a Dice score of 0.8063; in the second trial(e.g., element insertion implemented to a limited degree), the machinelearning model achieved a Dice score of 0.8309; in the third trial(e.g., element insertion implemented to a greater degree), the machinelearning model achieved a Dice score of 0.8795; and in the fourth trial(e.g., element insertion implemented to a greater degree andmodality-based modifications and geometric transformations implemented),the machine learning model achieve a Dice score of 0.9135. In otherwords, the machine learning model experienced significantperformance/efficacy improvements when trained on data sets generated byvarious embodiments of the subject innovation (e.g., the machinelearning model became more generalizable/robust, became more able tohandle difficult and/or unseen test cases and/or so on). That is, asshown by FIG. 10 , the herein-described techniques for increasing dataset variability (e.g., element/feature insertion, modality-basedvariation, geometric transformations) can independently and/orcollectively improve model performance. For at least these reasons,various embodiments of the subject innovation constitute a concrete andtangible technical improvement (e.g., they can improve the computationalperformance of machine learning models).

FIGS. 11-20 illustrate block diagrams of example, non-limiting imageaugmentations in accordance with one or more embodiments describedherein. In other words, FIGS. 11-20 depicts real-world examples ofvarious element insertions, various modality-based variations, and/orvarious geometric transformations that can be implemented in accordancewith various embodiments.

FIG. 11 depicts wires/cables 1102 (e.g., electrocardiogram leads,intravenous tubing, breathing tubes, and/or so on) and depicts how thewires/cables 1102 can be inserted into and/or superimposed on variousX-ray images 1104 in different locations, with different orientations,with different dimensions, with different thicknesses/intensities, withdifferent shapes, and/or so on.

FIG. 12 depicts masses 1202 (e.g., fluid-filled sacs, growths, cists,and/or so on) and depicts how the masses 1202 can be inserted intoand/or superimposed on various X-ray images 1204 in different locations,with different orientations, with different dimensions, with differentthicknesses/intensities, with different shapes, and/or so on.

FIG. 13 depicts mass 1302 (e.g., tumor, and/or so on) and depicts howthe mass 1302 can be inserted into and/or superimposed on various X-rayimages 1304 in different locations, with different orientations, withdifferent dimensions, with different thicknesses/intensities, withdifferent shapes, and/or so on.

FIG. 14 depicts stomach gas 1402 and depicts how the stomach gas 1402can be inserted into and/or superimposed on various X-ray images 1404 indifferent locations, with different orientations, with differentdimensions, with different thicknesses/intensities, with differentshapes, and/or so on. As explained above, note how the stomach gas 1402can be inserted into abdominal regions of the various X-ray images 1404rather than in chest regions of the various X-ray images 1404 (e.g.,stomach gas cannot form in the chest, but can form in the abdomen).

FIG. 15 depicts various X-ray images 1500 in which a gamma/radiationlevel is continuously varied. As shown, the gamma/radiation level ishighest in the upper-left X-ray images and is lowest in the lower-rightX-ray images.

FIG. 16 depicts various X-ray images 1600 in which a Gaussian noiselevel is continuously varied. As shown, the Gaussian noise level islowest in the upper-left X-ray images and is highest in the lower-rightX-ray images.

FIG. 17 depicts various X-ray images 1700 in which a Gaussian blur levelis continuously varied. As shown, the Gaussian blur level is lowest inthe upper-left X-ray images and is highest in the lower-right X-rayimages.

FIG. 18 depicts various X-ray images 1800 in which a contrast level iscontinuously varied. As shown, the contrast level is lowest in theupper-left X-ray images and is highest in the lower-right X-ray images.

FIG. 19 depicts various X-ray images 1900 in which a brightness level iscontinuously varied. As shown, the brightness level is lowest in theupper-left X-ray images and is highest in the lower-right X-ray images.

FIG. 20 depicts various X-ray images 2000 in which an exemplary opticaldistortion is continuously varied. As shown, the optical distortion ismost readily apparent in the upper-left and lower-right X-ray images.

It should be appreciated that the augmentations illustrated in FIGS.11-20 are exemplary and non-limiting. Any other suitable augmentationscan be implemented in various embodiments.

FIG. 21 illustrates a flow diagram of an example, non-limitingcomputer-implemented method 2100 that can facilitate synthetic trainingdata generation for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

In various embodiments, act 2102 can include generating, by a deviceoperatively coupled to a processor (e.g., 112), a set of preliminaryannotated training images (e.g., 204) based on an annotated source image(e.g., 104). In various aspects, a preliminary annotated training imagecan be formed by inserting at least one element of interest or at leastone background element (e.g., from 202) into the annotated source image.

In various instances, act 2104 can include generating, by the device(e.g., 114), a set of intermediate annotated training images (e.g., 504)based on the set of preliminary annotated training images. In variouscases, an intermediate annotated training image can be formed by varyingat least one modality-based characteristic (e.g., from 502) of apreliminary annotated training image.

In various aspects, act 2106 can include generating, by the device(e.g., 116), a set of deployable annotated training images (e.g., 704)based on the set of intermediate annotated training images. In variousinstances, a deployable annotated training image can be formed byvarying at least one geometric characteristic (e.g., by applying any of702) of an intermediate annotated training image.

In various cases, act 2108 can include training, by the device (e.g.,118), a machine learning model (e.g., 106) on the set of deployableannotated training images.

FIG. 22 illustrates a flow diagram of an example, non-limitingcomputer-implemented method 2200 that can facilitate synthetic trainingdata generation for improved machine learning model generalizability inaccordance with one or more embodiments described herein.

As explained above, the herein teachings regarding how to syntheticallyincrease training data variability are described, for ease ofexplanation, with respect to an imaging context (e.g., the machinelearning model 106 is configured to analyze one or more images).However, in various aspects, the described teachings can be applied inany suitable context that utilizes machine learning models (e.g., modelsthat analyze images, models that analyze sound recordings, and/or modelsthat analyze any other suitable type of data). In such cases, it shouldbe understood that the format of the source data (e.g., 104), thepreliminary training data (e.g., 204), the intermediate training data(e.g., 504), the deployable training data (e.g., 704), the elementcatalog 202, the modality-based characteristics 502, and/or thegeometric transformations 702 can depend upon the operational context(e.g., source/training images can be implemented if the machine learningmodel 106 is configured to analyze images; source/training soundrecordings can be implemented if the machine learning model 106 isconfigured to analyze sound recordings; the types and/or format ofinsertable elements, modifiable modality-based characteristics, and/ormathematical transformations can depend on the format of the data whichthe machine learning model 106 is configured to analyze; and/or so on).The computer-implemented method 2200 demonstrates this generalizability.

In various embodiments, act 2202 can include parametrizing, by a deviceoperatively coupled to a processor (e.g., 112), a first space ofpotential data features (e.g., 202). In various cases, these can includefeatures of interest and/or background features that can be insertedinto a data segment.

In various instances, act 2204 can include parametrizing, by the device(e.g., 114), a second space of potential modality-based data properties(e.g., 502). In various cases, these can include properties of a datasegment that are related to the particular device modality that was usedto capture and/or generate the data segment.

In various aspects, act 2206 can include parametrizing, by the device(e.g., 116), a third space of potential data transformations (e.g.,702). In various cases, these can include mathematical transformationsand/or operations that can be applied to a data segment.

In various embodiments, act 2208 can include receiving, by the device, asource data segment (e.g., 104) with an associated annotation.

In various instances, act 2210 can include generating, by the device(e.g., 112), a set of preliminary training data segments (e.g., 204),wherein a preliminary training data segment can be formed by inserting adata feature from the first space (e.g., 202) into the source datasegment. In various cases, this can include taking a first parametricsampling of values/states from the first space, and applying differentcombinations/permutations of the first parametric sampling to the sourcedata segment to generate the set of preliminary training data segments.

In various aspects, act 2212 can include generating, by the device(e.g., 114), a set of intermediate training data segments (e.g., 504),wherein an intermediate training data segment can be formed by varying amodality-based data property from the second space (e.g., 502) of apreliminary training data segment. In various cases, this can includetaking a second parametric sampling of values/states from the secondspace, and applying different combinations/permutations of the secondparametric sampling to the set of preliminary training data segments togenerate the set of intermediate training data segments.

In various embodiments, act 2214 can include generating, by the device(e.g., 116), a set of deployable training data segments (e.g., 704),wherein a deployable training data segment can be formed by applying adata transformation from the third space (e.g., 702) to an intermediatetraining data segment. In various cases, this can include taking a thirdparametric sampling of values/states from the third space, and applyingdifferent combinations/permutations of the third parametric sampling tothe set of intermediate training data segments to generate the set ofdeployable training data segments.

In various cases, a machine learning model can then be trained on thedet of deployable training data segments.

Various embodiments of the subject innovation can achieve theirtechnical benefits by parametrizing a simulation space. Specifically, invarious embodiments, it can be desired to train a machine learning modelon a source data segment. In various aspects, a simulation space can bedefined, where the simulation space can be considered as the domain ofpossible input data that can be fed to the machine learning model to betrained (e.g., for a machine learning model that is designed to analyzechest X-ray images, the simulation space can be the space of allpossible chest X-ray images having different anatomicalstructures/features, different brightness/contrast levels, differentdistortion levels, different orientations/angles, and/or otherwisedifferent image signatures that might be fed to the machine learningmodel; for a machine learning model that is designed to analyze voicerecordings, the simulation space can be the space of all possible voicerecordings having different volumes and/or loudness/pressure levels,different pitches, different tones, and/or otherwise different soundsignatures that might be fed to the machine learning model). In variousinstances, the source data segment can be considered as representingmerely one point within the simulation space (e.g., one particular chestX-ray image in the space of all possible chest X-rays images; oneparticular voice recording in the space of all possible voicerecordings). Various embodiments of the subject innovation canautomatically generate a plurality of deployable training data segmentsbased on the source data segment, such that many varied points withinthe simulation space are now represented by the plurality of deployabletraining data segments (e.g., the source data segment can be copied, andthe copies can be manipulated and/or modulated such that they havevarious different permutations/combinations of features and/orproperties so as to more fully represent the diversity of the simulationspan). Specifically, in various aspects, the simulation space can beparametrized by defining one or more modifiable parameters that span thesimulation space. As explained thoroughly herein, non-limiting examplesof such modifiable parameters can include data elements/features ofinterest and/or background data elements/features that are insertableinto the annotated source data segment, modality-based datacharacteristics/properties of the source data segment that can bemodulated, and/or mathematical transformations that can be applied tothe source data segment. In various cases, the plurality of deployabletraining data segments can be generated by starting with the source datasegment and by applying any suitable combinations and/or permutations ofvalues and/or states to the modifiable parameters. In various instances,the result can be that the plurality of deployable training datasegments broadly and/or widely sample the simulation space. In otherwords, the plurality of deployable training data segments can representa very large sampling and/or proportion of the simulation space (e.g.,the plurality of deployable training images can represent, capture,and/or approximate the diversity of values/states in the simulationspace). Training the machine learning model on the plurality ofdeployable training data segments can result in improvedperformance/efficacy as compared to training the machine learning modelon the source data segment alone.

For instance, consider the following exemplary parametrizationhierarchy. First, a simulation space can be defined (e.g., it can be thedomain of possible input data segments having different data signaturesthat can be received by the machine learning model in question). Next,broad augmentation subspaces can be defined within the simulation space,where an augmentation subspace contains one or more related, augmentableparameters. For instance, as explained herein, a first augmentationsubspace can be a space of insertable data elements/features, and theone or more related, augmentable parameters within the space ofinsertable data elements/features can include types of insertable dataelements/features (e.g., when the data involved are images, such typesof insertable data elements/features can include images of breathingtubes, images of pacemakers, images of implants, images of fluid sacs,images of lung growths, images of stomach gas), localizations of theinsertable data elements/features (e.g., different insertable images canbe inserted into different image locations), orientations of theinsertable data elements/features (e.g., different insertable images canbe inserted upside down, backwards, sideways), dimensions/intensities ofthe insertable data elements/features (e.g., different insertable imagescan be inserted with different sizes/shapes/thicknesses), and/or so on.As another example, a second augmentation subspace can be a space ofmodifiable modality-based characteristics, where the one or morerelated, augmentable parameters within the space of modifiablemodality-based characteristics include any suitable data segmentproperties that depend upon and/or that can be otherwise related to thedevice modality that generated and/or captured the source data segmentsin question (e.g., gamma/radiation level of an image, brightness levelof an image, contrast level of an image, blur level of an image, noiselevel of an image, texture of an image, device artifacts in an image).As yet another example, a third augmentation subspace can be a space ofmathematical transformations, where the one or more related, augmentableparameters within the space of mathematical transformations include anysuitable operations that can be applied to the data segments in question(e.g., image rotations, image reflections, image pans, image tilts,image zooms, image distortions). In various aspects, each of the one ormore related, augmentable parameters within each augmentation subspacecan vary over a corresponding continuous parametric range of valuesand/or states. For example, the gamma/radiation level of an image canvary continuously from a minimum value to a maximum value. Similarly,the contrast level of an image can vary continuously from a minimumvalue to a maximum value. In some cases, however, an augmentableparameter can have a corresponding discrete range of values and/orstates (e.g., a modality artifact parameter can include a statecorresponding to no depicted artifacts, a state corresponding to adepicted lens glare, a state corresponding to a depicted lens scratch, astate corresponding to both a depicted lens glare and a depicted lensscratch, and/or so on). In various aspects, a parametric sampling ofvalues/states from the continuous (and/or discrete) parametric range ofeach augmentable parameter can be taken (e.g., for a given data segment,an augmentable parameter can have any of a set of possiblevalues/states, and a parametric sampling for that augmentable parametercan be any suitable subset of the set of possible values/states). Forexample, a gamma/radiation level of an image can continuously vary froma minimum value (e.g., 1 unit) to a maximum value (e.g., 1000 units),and the parametric sampling can include the minimum value, the maximumvalue, and any suitable, regular step sizes and/or increments betweenthe minimum value and the maximum (e.g., the parametric sampling ofgamma level values can go from 1 to 1000 in steps/increments of 0.1). Invarious aspects, a source data segment can be converted into a pluralityof deployable training data segments by augmenting the source datasegment according to any suitable combinations and/or permutations ofsuch sampled parametric ranges of values/states (e.g., copies of thesource data segment can be made, and different copies can be modified soas to have/exhibit different permutations/combinations of values/statesfrom the sampled parametric ranges). Thus, the result can be that theplurality of deployable training data segments more adequately spanand/or represent the feature/property diversity of the simulation spacethan does the source data segment alone, and so training the machinelearning model on the plurality of deployable training data segments canyield better model performance as compared to conventional trainingtechniques.

FIG. 23 illustrates a block diagram of an example, non-limitingaugmentation space hierarchy 2300 that can facilitate synthetic trainingdata generation for improved machine learning model generalizability inaccordance with one or more embodiments described herein. In variousaspects, FIG. 23 can help to illustrate some of the aspects discussedabove.

In various embodiments, it can be desired to train a machine learningmodel (e.g., 106). Accordingly, in various instances, a simulation space2302 can be defined. In some cases, the simulation space 2302 can be thedomain of all possible input data segments that can be received and/oranalyzed by the machine learning model to be trained (e.g., if themachine learning model is configured to analyze brain CT scans, thesimulation space 2302 can be the domain of all possible brain CT scanshaving different brain shapes, different anatomical features/properties,different disease states, different pixel values, and/or so on).

In various aspects, the simulation space 2302 can be decomposed into aset of augmentation subspaces 2304. In various aspects, as shown, Xaugmentation subspaces (e.g., augmentation subspace 1 to augmentationsubspace X) can be defined for the simulation space 2302, for anysuitable integer X. In various cases, each augmentation subspace can beconsidered as a space of related and augmentable parameters thatcollectively make up the simulation space 2302. As explained thoroughlyabove, non-limiting examples of such augmentation subspaces can includean element/feature subspace (e.g., a collection of augmentableparameters that relate to data elements/features that are insertableinto data segments), a modality-based subspace (e.g., a collection ofaugmentable parameters that relate to settings of device modalities thatcapture/generate data segments and which can be modulated for datasegments), and/or a data transformation subspace (e.g., a collection ofaugmentable parameters that relate to mathematical operations that canbe applied to data segments).

In various aspects, each augmentation subspace can comprise a set ofaugmentable parameters. As shown, the augmentation subspace 1 cancomprise the set of augmentable parameters 2306 (e.g., augmentableparameter 1_1 to augmentable parameter 1_Y, for any suitable integer Y).Similarly, the augmentation subspace X can comprise the set ofaugmentable parameters 2308 (e.g., augmentable parameter X_1 toaugmentable parameter X_Y, for any suitable integer Y). Although FIG. 23shows both the set of augmentable parameters 2306 and the set ofaugmentable parameters 2308 as having the same number of parameters(e.g., Y), this is exemplary and non-limiting. In various aspects, eachset of augmentable parameters can have any suitable number ofaugmentable parameters (e.g., some sets having the same number ofparameters, some sets having different numbers of parameters, and/or soon). As thoroughly explained above, non-limiting examples of suchaugmentable parameters can include the following: an element/featureaugmentation subspace can include as augmentable parameterselement/feature type, element/feature localization, element/featuredimensions, element/feature orientations, element/feature intensities,and/or so on; a modality-based subspace can include as augmentableparameters a brightness level, a contrast level, a noise/blur level, aresolution level, device artifacts, and/or so on; a data transformationsubspace can include as augmentable parameters a reflection operation, arotation operation, a panning/tilting/zooming operation, a distortionoperation, and/or so on.

In various aspects, as shown, each augmentable parameter can have itsown parametric range of possible values/states. For instance, theaugmentable parameter 1_1 can have an associated parametric range ofpossible values/states for the augmentable parameter 1_1, theaugmentable parameter 1_Y can have a parametric range of possiblevalues/states for the augmentable parameter 1_Y, the augmentableparameter X_1 can have a parametric range of possible values/states forthe augmentable parameter X_1, the augmentable parameter X_Y can have aparametric range of possible values/states for the augmentable parameterX_Y, and/or so on. As a non-limiting example, an augmentable parameterin an element/feature subspace can be type, and the correspondingparametric range of possible values/states can be all the possible typesof elements/features which can be inserted into a data segment (e.g.,images of pacemakers, images of implants, images of breathing tubes,images of electrocardiogram leads, images of lung growths, images offluid sacs, images of stomach gas, and/or so on). As anothernon-limiting example, an augmentable parameter in an element/featuresubspace can be localization, and the corresponding parametric range ofpossible values/states can be all the possible locations within a datasegment where an element/feature can be inserted (e.g., top left of animage, bottom right of an image, middle of an image, and/or so on). Asstill another example, an augmentable parameter in an element/featuresubspace can be orientation, and the corresponding parametric range ofpossible values/states can be all the possible orientations which anelement/feature can have when inserted into a data segment (e.g., rightside up, upside down, backwards, sideways, tilted, and/or so on). Asanother example, an augmentable parameter in a modality-based subspacecan be brightness, and the corresponding parametric range of possiblevalues/states can be all the possible brightness levels which a datasegment can have (e.g., continuously ranging in magnitude from a minimumbrightness to a maximum brightness). As yet another example, anaugmentable parameter in a modality-based subspace can be contrast, andthe corresponding parametric range of possible values/states can be allthe possible contrast levels which a data segment can have (e.g.,continuously ranging in magnitude from a minimum contrast to a maximumcontrast). As still another example, an augmentable parameter in amodality-based subspace can be device artifacts, and the correspondingparametric range of possible values/states can be all the possibledevice artifacts which a data segment can have (e.g., lens glares ofvarying sizes/locations, lens scratches of varying sizes/locations,other lens occlusions like dust/dirt of varying sizes/locations,combinations of artifacts, no artifacts, and/or so on). As anotherexample, an augmentable parameter in a data transformation subspace canbe reflections, and the corresponding parametric range of possiblevalues/states can be all the possible reflections that can be applied toa data segment (e.g., horizontal reflections, vertical reflections,reflections about any other suitable axis, and/or so on). As yet anotherexample, an augmentable parameter in a data transformation subspace canbe rotations, and the corresponding parametric range of possiblevalues/states can be all the possible rotations that can be applied to adata segment (e.g., continuously ranging in magnitude from a minimumangular rotation to a maximum angular rotation). As still anotherexample, an augmentable parameter in a data transformation subspace canbe distortions, and the corresponding parametric range of possiblevalues/states can be all the possible distortions that can be applied toa data segment (e.g., barrel distortions of varying magnitude, mustachedistortions of varying magnitude, pincushion distortions of varyingmagnitude, combinations of distortions, no distortions, and/or so on).

In various aspects, the parametric range of possible values/states forthe augmentable parameter 1_1 and the parametric range of possiblevalues/states for the augmentable parameter 1_Y can be considered as aset of parametric ranges of possible values/states 2310, which setcorresponds to the set of augmentable parameters 2306. In variousinstances, the set of parametric ranges of possible values/states 2310can be considered as all the possible values/states that span theaugmentation subspace 1. Similarly, the parametric range of possiblevalues/states for the augmentable parameter X_1 and the parametric rangeof possible values/states for the augmentable parameter X_Y can beconsidered as a set of parametric ranges of possible values/states 2312,which set corresponds to the set of augmentable parameters 2308. Invarious instances, the set of parametric ranges of possiblevalues/states 2312 can be considered as all the possible values/statesthat span the augmentation subspace X. Therefore, in some cases, thesets of parametric ranges of possible values/states 2310 and 2312 cancollectively be considered as spanning the simulation space 2302.

In various embodiments, a sample can be taken of each parametric rangeof possible values/states. For instance, a sampled range ofvalues/states for the augmentable parameter 1_1 can be taken from theparametric range of possible values/states for the augmentable parameter1_1, a sampled range of values/states for the augmentable parameter 1_Ycan be taken from the parametric range of possible values/states for theaugmentable parameter 1_Y, a sampled range of values/states for theaugmentable parameter X_1 can be taken from the parametric range ofpossible values/states for the augmentable parameter X_1, the sampledrange of values/states for the augmentable parameter X_Y can be takenfrom the parametric range of possible values/states for the augmentableparameter X_Y, and/or so on. In various aspects, for a given parametricrange of possible values/states, a sampled range of values/states can beany suitable subset of the given parametric range of possiblevalues/states. In various instances, such sampled ranges ofvalues/states can be used to generate deployable training data segmentsas described herein. In other words, properties/characteristics oftraining data segments can be manipulated to take on any suitablecombinations/permutations of the values/states represented in thesampled ranges of values/states. In various aspects, the sampled rangeof values/states for the augmentable parameter 1_1 and the sampled rangeof values/states for the augmentable parameter 1_Y can be considered asa set of sampled ranges of values/states 2314, which set corresponds tothe set of parametric ranges of possible values/states 2310. Similarly,the sampled range of values/states for the augmentable parameter X_1 andthe sampled range of values/states for the augmentable parameter X_Y canbe considered as a set of sampled ranges of values/states 2316, whichset corresponds to the set of parametric ranges of possiblevalues/states 2312. In some cases, the sets of sampled ranges ofvalues/states 2314 and 2316 can be collectively considered as an overallset and/or collection of values/states that represents and/orapproximates the simulation space 2302 (e.g., that represents and/orapproximates the diversity and/or variability of data features, dataproperties, and/or data characteristics within the simulation space2302; in some cases, this can be a coarse and/or fine approximation ofthe diversity and/or variability of the simulation space 2302 dependingupon the cardinality, resolution, and/or step sizes of the sampledranges).

As explained thoroughly above, when deployable training data segmentsare synthetically generated based on the sets of sampled ranges ofvalues/states 2314 and 2316, the deployable training data segments canmore fully approximate and/or represent the variability and/or diversityof the simulation space 2302. Thus, training the machine learning modelof interest on such deployable training data segments can result inimproved model efficacy/performance as compared to traditional trainingtechniques.

In various aspects, embodiments of the subject innovation can beconsidered as a robust and/or methodical technique for decomposing asimulation space (e.g., 2302) into augmentation subspaces (e.g., 2304),for decomposing the augmentation subspaces into augmentable parameters(e.g., 2306, 2308), for defining parametric ranges of possiblevalues/states (e.g., 2310, 2312) for those augmentable parameters, forsampling (e.g., 2314, 2316) those parametric ranges of possiblevalues/states, and for applying those sampled parametric ranges totraining data segments such that those training data segments adequatelyspan, represent, and/or capture the variability and/or diversity of theoverall simulation space.

As explained above, in various aspects, embodiments of the subjectinnovation can update, change, and/or edit the parametrization of thesimulation space 2302 (e.g., by defining and/or creating new and/ordifferent augmentation subspaces in the set of augmentation subspaces2304, by defining and/or creating new and/or different augmentableparameters for each augmentation subspace, by altering the parametricranges of possible values/states for each augmentable parameter, and/orby taking different samples of the parametric ranges of possiblevalues/states for each augmentable parameter).

Although various embodiments of the subject innovation are describedherein as applying image/data augmentations in a specific order (e.g.,first element insertion, then modality-based modulation, and finallygeometric transformation), this is exemplary, non-limiting, and for easeof explanation. In various aspects, such image/data augmentations can beperformed in any suitable order.

Machine learning model generalizability can be an important aspect ofany artificial intelligence project. But generalizability can dependupon the availability and/or variety of annotated training data. Variousembodiments of the subject innovation provide for systems and/ortechniques that can synthetically generate varied training data based ona given piece of annotated training data. In various aspects,deterministic data augmentation can be applied as described herein tosynthetically generate such varied training data. Specifically,element/feature insertion, modality-based modulation, and geometrictransformations can be performed in any suitable order to syntheticallygenerate voluminous and varied training data. As explained herein,training a machine learning model on such synthetically generatedtraining data can result in significant performance/efficacyimprovements. This performance/efficacy improvement can be achievedbecause the disclosed data augmentations can cause the syntheticallygenerated training data to simulate and/or approximate real-worldvariability that the machine learning model is likely to encounterduring operation.

In various embodiments, an image from a source dataset can be selected.In various aspects, any suitable permutation and/or combination ofelement/feature insertions, modality-based variations, and/or geometrictransformations can be performed on the selected image to generate thedeployable training images. In various aspects, any suitableaugmentation policy/scheme can be implemented that controls how eachimage is augmented. In various aspects, each parameter of theaugmentation policy can have its own range of values to be applied(e.g., rotation between 0 degrees and 360 degrees, gamma between 50microwatts and 250 microwatts, and so on). In some cases, variousaugmentations can have an associated execution probability (e.g.,meaning that the augmentation can be performed for fewer than all theimages). In various aspects, any suitable augmentation policy/scheme canbe implemented so as to improve/simulate real-world data variability. Insome cases, different augmentation policies can be formulated based ondata dimensionality (e.g., different policies for one-dimensional,two-dimensional, three-dimensional, and/or so on).

As shown above, various embodiments of the subject innovation aredescribed with respect to the annotated source image 104. Specifically,various embodiments of the subject innovation can quickly andautomatically generate the set of deployable training images 704 basedon the annotated source image 104, where the set of deployable trainingimages 704 can be used to facilitate supervised training of the machinelearning model 106. However, in various other embodiments, thedeployable training images 704 can be generated based on an unannotatedsource image (not shown in the figures). In such case, the deployabletraining images 704 could lack annotations/labels and could thus be usedto facilitate unsupervised training and/or reinforcement learning of themachine learning model 106. In other words, those having ordinary skillin the art will appreciate that the herein teachings can be applied toan annotated source image as well as an unannotated source image.

In order to provide additional context for various embodiments describedherein, FIG. 24 and the following discussion are intended to provide abrief, general description of a suitable computing environment 2400 inwhich the various embodiments of the embodiment described herein can beimplemented. While the embodiments have been described above in thegeneral context of computer-executable instructions that can run on oneor more computers, those skilled in the art will recognize that theembodiments can be also implemented in combination with other programmodules and/or as a combination of hardware and software.

Generally, program modules include routines, programs, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the inventive methods can be practiced with other computer systemconfigurations, including single-processor or multiprocessor computersystems, minicomputers, mainframe computers, Internet of Things (IoT)devices, distributed computing systems, as well as personal computers,hand-held computing devices, microprocessor-based or programmableconsumer electronics, and the like, each of which can be operativelycoupled to one or more associated devices.

The illustrated embodiments of the embodiments herein can be alsopracticed in distributed computing environments where certain tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules can be located in both local and remote memory storage devices.

Computing devices typically include a variety of media, which caninclude computer-readable storage media, machine-readable storage media,and/or communications media, which two terms are used herein differentlyfrom one another as follows. Computer-readable storage media ormachine-readable storage media can be any available storage media thatcan be accessed by the computer and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable storage media or machine-readablestorage media can be implemented in connection with any method ortechnology for storage of information such as computer-readable ormachine-readable instructions, program modules, structured data orunstructured data.

Computer-readable storage media can include, but are not limited to,random access memory (RAM), read only memory (ROM), electricallyerasable programmable read only memory (EEPROM), flash memory or othermemory technology, compact disk read only memory (CD-ROM), digitalversatile disk (DVD), Blu-ray disc (BD) or other optical disk storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, solid state drives or other solid statestorage devices, or other tangible and/or non-transitory media which canbe used to store desired information. In this regard, the terms“tangible” or “non-transitory” herein as applied to storage, memory orcomputer-readable media, are to be understood to exclude onlypropagating transitory signals per se as modifiers and do not relinquishrights to all standard storage, memory or computer-readable media thatare not only propagating transitory signals per se.

Computer-readable storage media can be accessed by one or more local orremote computing devices, e.g., via access requests, queries or otherdata retrieval protocols, for a variety of operations with respect tothe information stored by the medium.

Communications media typically embody computer-readable instructions,data structures, program modules or other structured or unstructureddata in a data signal such as a modulated data signal, e.g., a carrierwave or other transport mechanism, and includes any information deliveryor transport media. The term “modulated data signal” or signals refersto a signal that has one or more of its characteristics set or changedin such a manner as to encode information in one or more signals. By wayof example, and not limitation, communication media include wired media,such as a wired network or direct-wired connection, and wireless mediasuch as acoustic, RF, infrared and other wireless media.

With reference again to FIG. 24 , the example environment 2400 forimplementing various embodiments of the aspects described hereinincludes a computer 2402, the computer 2402 including a processing unit2404, a system memory 2406 and a system bus 2408. The system bus 2408couples system components including, but not limited to, the systemmemory 2406 to the processing unit 2404. The processing unit 2404 can beany of various commercially available processors. Dual microprocessorsand other multi-processor architectures can also be employed as theprocessing unit 2404.

The system bus 2408 can be any of several types of bus structure thatcan further interconnect to a memory bus (with or without a memorycontroller), a peripheral bus, and a local bus using any of a variety ofcommercially available bus architectures. The system memory 2406includes ROM 2410 and RAM 2412. A basic input/output system (BIOS) canbe stored in a non-volatile memory such as ROM, erasable programmableread only memory (EPROM), EEPROM, which BIOS contains the basic routinesthat help to transfer information between elements within the computer2402, such as during startup. The RAM 2412 can also include a high-speedRAM such as static RAM for caching data.

The computer 2402 further includes an internal hard disk drive (HDD)2414 (e.g., EIDE, SATA), one or more external storage devices 2416(e.g., a magnetic floppy disk drive (FDD) 2416, a memory stick or flashdrive reader, a memory card reader, etc.) and a drive 2420, e.g., suchas a solid state drive, an optical disk drive, which can read or writefrom a disk 2422, such as a CD-ROM disc, a DVD, a BD, etc.Alternatively, where a solid state drive is involved, disk 2422 wouldnot be included, unless separate. While the internal HDD 2414 isillustrated as located within the computer 2402, the internal HDD 2414can also be configured for external use in a suitable chassis (notshown). Additionally, while not shown in environment 2400, a solid statedrive (SSD) could be used in addition to, or in place of, an HDD 2414.The HDD 2414, external storage device(s) 2416 and drive 2420 can beconnected to the system bus 2408 by an HDD interface 2424, an externalstorage interface 2426 and a drive interface 2428, respectively. Theinterface 2424 for external drive implementations can include at leastone or both of Universal Serial Bus (USB) and Institute of Electricaland Electronics Engineers (IEEE) 1394 interface technologies. Otherexternal drive connection technologies are within contemplation of theembodiments described herein.

The drives and their associated computer-readable storage media providenonvolatile storage of data, data structures, computer-executableinstructions, and so forth. For the computer 2402, the drives andstorage media accommodate the storage of any data in a suitable digitalformat. Although the description of computer-readable storage mediaabove refers to respective types of storage devices, it should beappreciated by those skilled in the art that other types of storagemedia which are readable by a computer, whether presently existing ordeveloped in the future, could also be used in the example operatingenvironment, and further, that any such storage media can containcomputer-executable instructions for performing the methods describedherein.

A number of program modules can be stored in the drives and RAM 2412,including an operating system 2430, one or more application programs2432, other program modules 2434 and program data 2436. All or portionsof the operating system, applications, modules, and/or data can also becached in the RAM 2412. The systems and methods described herein can beimplemented utilizing various commercially available operating systemsor combinations of operating systems.

Computer 2402 can optionally comprise emulation technologies. Forexample, a hypervisor (not shown) or other intermediary can emulate ahardware environment for operating system 2430, and the emulatedhardware can optionally be different from the hardware illustrated inFIG. 24 . In such an embodiment, operating system 2430 can comprise onevirtual machine (VM) of multiple VMs hosted at computer 2402.Furthermore, operating system 2430 can provide runtime environments,such as the Java runtime environment or the .NET framework, forapplications 2432. Runtime environments are consistent executionenvironments that allow applications 2432 to run on any operating systemthat includes the runtime environment. Similarly, operating system 2430can support containers, and applications 2432 can be in the form ofcontainers, which are lightweight, standalone, executable packages ofsoftware that include, e.g., code, runtime, system tools, systemlibraries and settings for an application.

Further, computer 2402 can be enable with a security module, such as atrusted processing module (TPM). For instance with a TPM, bootcomponents hash next in time boot components, and wait for a match ofresults to secured values, before loading a next boot component. Thisprocess can take place at any layer in the code execution stack ofcomputer 2402, e.g., applied at the application execution level or atthe operating system (OS) kernel level, thereby enabling security at anylevel of code execution.

A user can enter commands and information into the computer 2402 throughone or more wired/wireless input devices, e.g., a keyboard 2438, a touchscreen 2440, and a pointing device, such as a mouse 2442. Other inputdevices (not shown) can include a microphone, an infrared (IR) remotecontrol, a radio frequency (RF) remote control, or other remote control,a joystick, a virtual reality controller and/or virtual reality headset,a game pad, a stylus pen, an image input device, e.g., camera(s), agesture sensor input device, a vision movement sensor input device, anemotion or facial detection device, a biometric input device, e.g.,fingerprint or iris scanner, or the like. These and other input devicesare often connected to the processing unit 2404 through an input deviceinterface 2444 that can be coupled to the system bus 2408, but can beconnected by other interfaces, such as a parallel port, an IEEE 1394serial port, a game port, a USB port, an IR interface, a BLUETOOTH®interface, etc.

A monitor 2446 or other type of display device can be also connected tothe system bus 2408 via an interface, such as a video adapter 2448. Inaddition to the monitor 2446, a computer typically includes otherperipheral output devices (not shown), such as speakers, printers, etc.

The computer 2402 can operate in a networked environment using logicalconnections via wired and/or wireless communications to one or moreremote computers, such as a remote computer(s) 2450. The remotecomputer(s) 2450 can be a workstation, a server computer, a router, apersonal computer, portable computer, microprocessor-based entertainmentappliance, a peer device or other common network node, and typicallyincludes many or all of the elements described relative to the computer2402, although, for purposes of brevity, only a memory/storage device2452 is illustrated. The logical connections depicted includewired/wireless connectivity to a local area network (LAN) 2454 and/orlarger networks, e.g., a wide area network (WAN) 2456. Such LAN and WANnetworking environments are commonplace in offices and companies, andfacilitate enterprise-wide computer networks, such as intranets, all ofwhich can connect to a global communications network, e.g., theInternet.

When used in a LAN networking environment, the computer 2402 can beconnected to the local network 2454 through a wired and/or wirelesscommunication network interface or adapter 2458. The adapter 2458 canfacilitate wired or wireless communication to the LAN 2454, which canalso include a wireless access point (AP) disposed thereon forcommunicating with the adapter 2458 in a wireless mode.

When used in a WAN networking environment, the computer 2402 can includea modem 2460 or can be connected to a communications server on the WAN2456 via other means for establishing communications over the WAN 2456,such as by way of the Internet. The modem 2460, which can be internal orexternal and a wired or wireless device, can be connected to the systembus 2408 via the input device interface 2444. In a networkedenvironment, program modules depicted relative to the computer 2402 orportions thereof, can be stored in the remote memory/storage device2452. It will be appreciated that the network connections shown areexample and other means of establishing a communications link betweenthe computers can be used.

When used in either a LAN or WAN networking environment, the computer2402 can access cloud storage systems or other network-based storagesystems in addition to, or in place of, external storage devices 2416 asdescribed above, such as but not limited to a network virtual machineproviding one or more aspects of storage or processing of information.Generally, a connection between the computer 2402 and a cloud storagesystem can be established over a LAN 2454 or WAN 2456 e.g., by theadapter 2458 or modem 2460, respectively. Upon connecting the computer2402 to an associated cloud storage system, the external storageinterface 2426 can, with the aid of the adapter 2458 and/or modem 2460,manage storage provided by the cloud storage system as it would othertypes of external storage. For instance, the external storage interface2426 can be configured to provide access to cloud storage sources as ifthose sources were physically connected to the computer 2402.

The computer 2402 can be operable to communicate with any wirelessdevices or entities operatively disposed in wireless communication,e.g., a printer, scanner, desktop and/or portable computer, portabledata assistant, communications satellite, any piece of equipment orlocation associated with a wirelessly detectable tag (e.g., a kiosk,news stand, store shelf, etc.), and telephone. This can include WirelessFidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, thecommunication can be a predefined structure as with a conventionalnetwork or simply an ad hoc communication between at least two devices.

FIG. 25 is a schematic block diagram of a sample computing environment2500 with which the disclosed subject matter can interact. The samplecomputing environment 2500 includes one or more client(s) 2510. Theclient(s) 2510 can be hardware and/or software (e.g., threads,processes, computing devices). The sample computing environment 2500also includes one or more server(s) 2530. The server(s) 2530 can also behardware and/or software (e.g., threads, processes, computing devices).The servers 2530 can house threads to perform transformations byemploying one or more embodiments as described herein, for example. Onepossible communication between a client 2510 and a server 2530 can be inthe form of a data packet adapted to be transmitted between two or morecomputer processes. The sample computing environment 2500 includes acommunication framework 2550 that can be employed to facilitatecommunications between the client(s) 2510 and the server(s) 2530. Theclient(s) 2510 are operably connected to one or more client datastore(s) 2520 that can be employed to store information local to theclient(s) 2510. Similarly, the server(s) 2530 are operably connected toone or more server data store(s) 2540 that can be employed to storeinformation local to the servers 2530.

The present invention may be a system, a method, an apparatus and/or acomputer program product at any possible technical detail level ofintegration. The computer program product can include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention. The computer readable storage medium can be atangible device that can retain and store instructions for use by aninstruction execution device. The computer readable storage medium canbe, for example, but is not limited to, an electronic storage device, amagnetic storage device, an optical storage device, an electromagneticstorage device, a semiconductor storage device, or any suitablecombination of the foregoing. A non-exhaustive list of more specificexamples of the computer readable storage medium can also include thefollowing: a portable computer diskette, a hard disk, a random accessmemory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), a static random access memory(SRAM), a portable compact disc read-only memory (CD-ROM), a digitalversatile disk (DVD), a memory stick, a floppy disk, a mechanicallyencoded device such as punch-cards or raised structures in a groovehaving instructions recorded thereon, and any suitable combination ofthe foregoing. A computer readable storage medium, as used herein, isnot to be construed as being transitory signals per se, such as radiowaves or other freely propagating electromagnetic waves, electromagneticwaves propagating through a waveguide or other transmission media (e.g.,light pulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network can comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device. Computer readable programinstructions for carrying out operations of the present invention can beassembler instructions, instruction-set-architecture (ISA) instructions,machine instructions, machine dependent instructions, microcode,firmware instructions, state-setting data, configuration data forintegrated circuitry, or either source code or object code written inany combination of one or more programming languages, including anobject oriented programming language such as Smalltalk, C++, or thelike, and procedural programming languages, such as the “C” programminglanguage or similar programming languages. The computer readable programinstructions can execute entirely on the user's computer, partly on theuser's computer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer can beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection can be made to an external computer (for example, through theInternet using an Internet Service Provider). In some embodiments,electronic circuitry including, for example, programmable logiccircuitry, field-programmable gate arrays (FPGA), or programmable logicarrays (PLA) can execute the computer readable program instructions byutilizing state information of the computer readable programinstructions to personalize the electronic circuitry, in order toperform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions. These computer readable programinstructions can be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. These computer readable program instructions can also be storedin a computer readable storage medium that can direct a computer, aprogrammable data processing apparatus, and/or other devices to functionin a particular manner, such that the computer readable storage mediumhaving instructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks. Thecomputer readable program instructions can also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational acts to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowcharts and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams can represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks can occur out of theorder noted in the Figures. For example, two blocks shown in successioncan, in fact, be executed substantially concurrently, or the blocks cansometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the subject matter has been described above in the general contextof computer-executable instructions of a computer program product thatruns on a computer and/or computers, those skilled in the art willrecognize that this disclosure also can or can be implemented incombination with other program modules. Generally, program modulesinclude routines, programs, components, data structures, etc. thatperform particular tasks and/or implement particular abstract datatypes. Moreover, those skilled in the art will appreciate that theinventive computer-implemented methods can be practiced with othercomputer system configurations, including single-processor ormultiprocessor computer systems, mini-computing devices, mainframecomputers, as well as computers, hand-held computing devices (e.g., PDA,phone), microprocessor-based or programmable consumer or industrialelectronics, and the like. The illustrated aspects can also be practicedin distributed computing environments in which tasks are performed byremote processing devices that are linked through a communicationsnetwork. However, some, if not all aspects of this disclosure can bepracticed on stand-alone computers. In a distributed computingenvironment, program modules can be located in both local and remotememory storage devices.

As used in this application, the terms “component,” “system,”“platform,” “interface,” and the like, can refer to and/or can include acomputer-related entity or an entity related to an operational machinewith one or more specific functionalities. The entities disclosed hereincan be either hardware, a combination of hardware and software,software, or software in execution. For example, a component can be, butis not limited to being, a process running on a processor, a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on aserver and the server can be a component. One or more components canreside within a process and/or thread of execution and a component canbe localized on one computer and/or distributed between two or morecomputers. In another example, respective components can execute fromvarious computer readable media having various data structures storedthereon. The components can communicate via local and/or remoteprocesses such as in accordance with a signal having one or more datapackets (e.g., data from one component interacting with anothercomponent in a local system, distributed system, and/or across a networksuch as the Internet with other systems via the signal). As anotherexample, a component can be an apparatus with specific functionalityprovided by mechanical parts operated by electric or electroniccircuitry, which is operated by a software or firmware applicationexecuted by a processor. In such a case, the processor can be internalor external to the apparatus and can execute at least a part of thesoftware or firmware application. As yet another example, a componentcan be an apparatus that provides specific functionality throughelectronic components without mechanical parts, wherein the electroniccomponents can include a processor or other means to execute software orfirmware that confers at least in part the functionality of theelectronic components. In an aspect, a component can emulate anelectronic component via a virtual machine, e.g., within a cloudcomputing system.

In addition, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom context, “X employs A or B” is intended to mean any of the naturalinclusive permutations. That is, if X employs A; X employs B; or Xemploys both A and B, then “X employs A or B” is satisfied under any ofthe foregoing instances. Moreover, articles “a” and “an” as used in thesubject specification and annexed drawings should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. As used herein, the terms “example”and/or “exemplary” are utilized to mean serving as an example, instance,or illustration. For the avoidance of doubt, the subject matterdisclosed herein is not limited by such examples. In addition, anyaspect or design described herein as an “example” and/or “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs, nor is it meant to preclude equivalent exemplarystructures and techniques known to those of ordinary skill in the art.

As it is employed in the subject specification, the term “processor” canrefer to substantially any computing processing unit or devicecomprising, but not limited to, single-core processors;single-processors with software multithread execution capability;multi-core processors; multi-core processors with software multithreadexecution capability; multi-core processors with hardware multithreadtechnology; parallel platforms; and parallel platforms with distributedshared memory. Additionally, a processor can refer to an integratedcircuit, an application specific integrated circuit (ASIC), a digitalsignal processor (DSP), a field programmable gate array (FPGA), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Further, processors can exploit nano-scalearchitectures such as, but not limited to, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of user equipment. A processor can also beimplemented as a combination of computing processing units. In thisdisclosure, terms such as “store,” “storage,” “data store,” datastorage,” “database,” and substantially any other information storagecomponent relevant to operation and functionality of a component areutilized to refer to “memory components,” entities embodied in a“memory,” or components comprising a memory. It is to be appreciatedthat memory and/or memory components described herein can be eithervolatile memory or nonvolatile memory, or can include both volatile andnonvolatile memory. By way of illustration, and not limitation,nonvolatile memory can include read only memory (ROM), programmable ROM(PROM), electrically programmable ROM (EPROM), electrically erasable ROM(EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g.,ferroelectric RAM (FeRAM). Volatile memory can include RAM, which canact as external cache memory, for example. By way of illustration andnot limitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM),direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), andRambus dynamic RAM (RDRAM). Additionally, the disclosed memorycomponents of systems or computer-implemented methods herein areintended to include, without being limited to including, these and anyother suitable types of memory.

What has been described above include mere examples of systems andcomputer-implemented methods. It is, of course, not possible to describeevery conceivable combination of components or computer-implementedmethods for purposes of describing this disclosure, but one of ordinaryskill in the art can recognize that many further combinations andpermutations of this disclosure are possible. Furthermore, to the extentthat the terms “includes,” “has,” “possesses,” and the like are used inthe detailed description, claims, appendices and drawings such terms areintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments. The terminologyused herein was chosen to best explain the principles of theembodiments, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

Further aspects of various embodiments of the subject claimed innovationare provided in the subject matter that follows:

1. A system, comprising: a processor that executes computer-executablecomponents stored in a memory, the computer-executable componentscomprising: an element augmentation component that generates a set ofpreliminary annotated training images based on an annotated sourceimage, wherein a preliminary annotated training image is formed byinserting at least one element of interest or at least one backgroundelement into the annotated source image; a modality augmentationcomponent that generates a set of intermediate annotated training imagesbased on the set of preliminary annotated training images, wherein anintermediate annotated training image is formed by varying at least onemodality-based characteristic of a preliminary annotated training image;and a geometry augmentation component that generates a set of deployableannotated training images based on the set of intermediate annotatedtraining images, wherein a deployable annotated training image is formedby varying at least one geometric characteristic of an intermediateannotated training image.

2. The system of any preceding clause, wherein the computer-executablecomponents further comprise: a training component that trains a machinelearning model on the set of deployable annotated training images.

3. The system of any preceding clause, wherein the element augmentationcomponent maintains an element catalog that lists a set of images ofpossible elements of interest and that lists a set of images of possiblebackground elements that are insertable into the annotated source image,wherein the modality augmentation component maintains a list ofmodality-based characteristics that are modifiable in the preliminarytraining images, and wherein the geometry augmentation componentmaintains a list of geometric transformations that are appliable to theintermediate training images.

4. The system of any preceding clause, wherein the element augmentationcomponent updates the element catalog by including within the elementcatalog a new image of an element of interest or a new image of abackground element, wherein the modality augmentation component updatesthe list of modality-based characteristics by including within the listof modality-based characteristics new image properties that relate todevice modality, and wherein the geometry augmentation component updatesthe list of geometric transformations by including within the list ofgeometric transformations new operations that are appliable to images.

5. The system of any preceding clause, wherein the at least one elementof interest or the at least one background element is medical equipmentor a biological symptom manifestation.

6. The system of any preceding clause, wherein the element augmentationcomponent randomly localizes the at least one element of interest or theat least one background element in a range of biologically-possiblelocations within the annotated source image.

7. The system of any preceding clause, wherein the varying the at leastone modality-based characteristic includes varying an image gamma level,varying an image blur level, varying an image brightness level, varyingan image contrast level, varying an image noise level, varying an imagetexture, varying an image resolution, varying an image field of view, orapplying a modality artifact.

8. The system of any preceding clause, wherein the varying the at leastone geometric characteristic includes rotating about an image axis,reflecting about an image axis, image magnifying, image panning, imagetilting, or image distorting.

9. A computer-implemented method, comprising: generating, by a deviceoperatively coupled to a processor, a set of preliminary annotatedtraining images based on an annotated source image, wherein apreliminary annotated training image is formed by inserting at least oneelement of interest or at least one background element into theannotated source image; generating, by the device, a set of intermediateannotated training images based on the set of preliminary annotatedtraining images, wherein an intermediate annotated training image isformed by varying at least one modality-based characteristic of apreliminary annotated training image; and generating, by the device, aset of deployable annotated training images based on the set ofintermediate annotated training images, wherein a deployable annotatedtraining image is formed by varying at least one geometriccharacteristic of an intermediate annotated training image.

10. The computer-implemented method of any preceding clause, furthercomprising: training, by the device, a machine learning model on the setof deployable annotated training images.

11. The computer-implemented method of any preceding clause, furthercomprising: maintaining, by the device, an element catalog that lists aset of images of possible elements of interest and that lists a set ofimages of possible background elements that are insertable into theannotated source image; maintaining, by the device, a list ofmodality-based characteristics that are modifiable in the preliminarytraining images; and maintaining, by the device, a list of geometrictransformations that are appliable to the intermediate training images.

12. The computer-implemented method of any preceding clause, furthercomprising: updating, by the device, the element catalog by includingwithin the element catalog a new image of an element of interest or anew image of a background element; updating, by the device, the list ofmodality-based characteristics by including within the list ofmodality-based characteristics new image properties that relate todevice modality; and updating, by the device, the list of geometrictransformations by including within the list of geometrictransformations new operations that are appliable to images.

13. The computer-implemented method of any preceding clause, wherein theat least one element of interest or the at least one background elementis medical equipment or a biological symptom manifestation.

14. The computer-implemented method of any preceding clause, furthercomprising: randomly localizing, by the device, the at least one elementof interest or the at least one background element in a range ofbiologically-possible locations within the annotated source image.

15. The computer-implemented method of any preceding clause, wherein thevarying the at least one modality-based characteristic includes varyingan image gamma level, varying an image blur level, varying an imagebrightness level, varying an image contrast level, varying an imagenoise level, varying an image texture, varying an image resolution,varying an image field of view, or applying a modality artifact.

16. The computer-implemented method of any preceding clause, wherein thevarying the at least one geometric characteristic includes rotatingabout an image axis, reflecting about an image axis, imagemagnification, image panning, image tilting, or image distortion.

17. A computer program product for facilitating synthetic training datageneration for improved machine learning generalizability, the computerprogram product comprising a computer readable memory having programinstructions embodied therewith, the program instructions executable bya processor to cause the processor to: parametrize a simulation space ofdata segments by defining a set of augmentation subspaces, wherein eachaugmentation subspace comprises a corresponding set of augmentableparameters, and wherein each augmentable parameter has a correspondingparametric range of possible values or states; receive a source datasegment; for each augmentable parameter, sample a parametric range ofpossible values or states corresponding to the augmentable parameter,thereby yielding a collection of sampled ranges of values or states thatrepresents the simulation space; and generate a set of training datasegments by applying the collection of sampled ranges of values orstates to copies of the source data segment.

18. The computer program product of any preceding clause, wherein theprogram instructions are further executable to cause the processor to:train a machine learning model on the set of training data segments.

19. The computer program product of any preceding clause, wherein theprogram instructions are further executable to cause the processor to:update the parametrization of the simulation space by defining newaugmentation subspaces.

20. The computer program product of any preceding claim, wherein theprogram instructions are further executable to cause the processor to:update the parametrization of the simulation space by defining newaugmentable parameters within the set of augmentation subspaces.

What is claimed is:
 1. A system, comprising: a processor that executescomputer-executable instructions stored in a memory, which causes theprocessor to: access an annotated source image; generate a set ofpreliminary annotated training images based on the annotated sourceimage, wherein each preliminary annotated training image is formed byinserting a respective permutation of visual objects into the annotatedsource image, wherein such visual objects include medical equipment orbiological symptoms; generate a set of intermediate annotated trainingimages based on the set of preliminary annotated training images,wherein each intermediate annotated training image is formed by applyinga respective permutation of modality characteristic variations to arespective preliminary annotated training image, wherein such modalitycharacteristic variations include changes to image properties thatdepend upon settings or parameters of a medical imaging device thatcaptured or generated the annotated source image; and generate a set ofdeployable annotated training images based on the set of intermediateannotated training images, wherein each deployable annotated trainingimage is formed by applying a respective permutation of geometricvariations to a respective intermediate annotated training image,wherein such geometric variations include spatial transformations ofimage pixel grids.
 2. The system of claim 1, wherein execution of thecomputer-executable instructions further causes the processor to: traina machine learning model on the set of deployable annotated trainingimages.
 3. The system of claim 1, wherein the system maintains anelement catalog that lists a set of images of possible visual objectsthat are insertable into the annotated source image, wherein the systemmaintains a list of modality characteristics that are modifiable in thepreliminary training images, and wherein the system maintains a list ofgeometric transformations that are appliable to the intermediatetraining images.
 4. The system of claim 3, wherein the processor updatesthe element catalog by including within the element catalog a new imageof a visual object, wherein the processor updates the list of modalitycharacteristics by including within the list of modality characteristicsnew image properties that relate to device modality, and wherein theprocessor updates the list of geometric transformations by includingwithin the list of geometric transformations new spatial operations thatare appliable to images.
 5. The system of claim 2, wherein the visualobjects are objects of interest which the machine learning model isconfigured to detect.
 6. The system of claim 1, wherein the processorrandomly localizes the visual objects in a range ofbiologically-possible locations within the annotated source image. 7.The system of claim 1, wherein the applying a respective permutation ofmodality characteristic variations includes varying an image gammalevel, varying an image blur level, varying an image brightness level,varying an image contrast level, varying an image noise level, varyingan image texture, varying an image resolution, varying an image field ofview, or applying a modality artifact.
 8. The system of claim 1, whereinthe applying a respective permutation of geometric variations includesrotating about an image axis, reflecting about an image axis, imagemagnifying, image panning, image tilting, or image distorting.
 9. Acomputer-implemented method, comprising: accessing, by a deviceoperatively coupled to a processor, an annotated source image;generating, by the device, a set of preliminary annotated trainingimages based on the annotated source image, wherein each preliminaryannotated training image is formed by inserting a respective permutationof visual objects into the annotated source image, wherein such visualobjects include medical equipment or biological symptoms; generating, bythe device, a set of intermediate annotated training images based on theset of preliminary annotated training images, wherein each intermediateannotated training image is formed by applying a respective permutationof modality characteristic variations to a respective preliminaryannotated training image, wherein such modality characteristicvariations include changes to image properties that depend upon settingsor parameters of a medical imaging device that captured or generated theannotated source image; and generating, by the device, a set ofdeployable annotated training images based on the set of intermediateannotated training images, wherein each deployable annotated trainingimage is formed by applying a respective permutation of geometricvariations to a respective intermediate annotated training image,wherein such geometric variations include spatial transformations ofimage pixel grids.
 10. The computer-implemented method of claim 9,further comprising: training, by the device, a machine learning model onthe set of deployable annotated training images.
 11. Thecomputer-implemented method of claim 9, further comprising: maintaining,by the device, an element catalog that lists a set of images of possiblevisual objects that are insertable into the annotated source image;maintaining, by the device, a list of modality characteristics that aremodifiable in the preliminary training images; and maintaining, by thedevice, a list of geometric transformations that are appliable to theintermediate training images.
 12. The computer-implemented method ofclaim 11, further comprising: updating, by the device, the elementcatalog by including within the element catalog a new image of a visualobject; updating, by the device, the list of modality characteristics byincluding within the list of modality characteristics new imageproperties that relate to device modality; and updating, by the device,the list of geometric transformations by including within the list ofgeometric transformations new spatial operations that are appliable toimages.
 13. The computer-implemented method of claim 10, wherein thevisual objects are objects of interest which the machine learning modelis configured to detect.
 14. The computer-implemented method of claim 9,further comprising: randomly localizing, by the device, the visualobjects in a range of biologically-possible locations within theannotated source image.
 15. The computer-implemented method of claim 9,wherein the applying a respective permutation of modality characteristicvariations includes varying an image gamma level, varying an image blurlevel, varying an image brightness level, varying an image contrastlevel, varying an image noise level, varying an image texture, varyingan image resolution, varying an image field of view, or applying amodality artifact.
 16. The computer-implemented method of claim 9,wherein the applying a respective permutation of geometric variationsincludes rotating about an image axis, reflecting about an image axis,image magnifying, image panning, image tilting, or image distorting. 17.A non-transitory computer program product for facilitating synthetictraining data generation for improved machine learning generalizability,the non-transitory computer program product comprising a computerreadable memory having program instructions embodied therewith, theprogram instructions executable by a processor to cause the processorto: access an annotated source image; generate a set of preliminaryannotated training images based on the annotated source image, whereineach preliminary annotated training image is formed by inserting arespective permutation of visual objects into the annotated sourceimage, wherein such visual objects include medical equipment orbiological symptoms; generate a set of intermediate annotated trainingimages based on the set of preliminary annotated training images,wherein each intermediate annotated training image is formed by applyinga respective permutation of modality characteristic variations to arespective preliminary annotated training image, wherein such modalitycharacteristic variations include changes to image properties thatdepend upon settings or parameters of a medical imaging device thatcaptured or generated the annotated source image; and generate a set ofdeployable annotated training images based on the set of intermediateannotated training images, wherein each deployable annotated trainingimage is formed by applying a respective permutation of geometricvariations to a respective intermediate annotated training image,wherein such geometric variations include spatial transformations ofimage pixel grids.
 18. The non-transitory computer program product ofclaim 17, wherein the program instructions are further executable tocause the processor to: train a machine learning model on the set ofdeployable annotated training images.
 19. The non-transitory computerprogram product of claim 17, wherein the program instructions arefurther executable to cause the processor to: maintain an elementcatalog that lists a set of images of possible visual objects that areinsertable into the annotated source image; maintain a list of modalitycharacteristics that are modifiable in the preliminary training images;and maintain a list of geometric transformations that are appliable tothe intermediate training images.
 20. The non-transitory computerprogram product of claim 19, wherein the program instructions arefurther executable to cause the processor to: update the element catalogby including within the element catalog a new image of a visual object;update the list of modality characteristics by including within the listof modality characteristics new image properties that relate to devicemodality; and update the list of geometric transformations by includingwithin the list of geometric transformations new spatial operations thatare appliable to images.